Five Times Pixbu’s Tests Measured the Wrong Thing

I’m a solo dev shipping Pixbu — a self-care app where a pixel creature grows as you look after yourself — for Shipaton 2026. It’s live on both stores in 6 languages with 1,889 automated tests. This post isn’t about the features. It’s about a habit that saved the launch: whenever I caught myself knowing something about my own app, I measured it instead. I was wrong often enough that it stopped being embarrassing and started being a process. Pixbu’s creature evolves through six stages. Each stage should be visibly bigger than the last — that’s the entire emotional payoff of the app. I replaced the whole art set. Tests stayed green. Ship it, right? Then I actually counted the drawn pixels in each stage: baby 6572 → teen 5627 → adult 5840 → elder 6244 → mythic 6338. The creature shrank 14% at the first evolution — the transition users see most. The single moment the app exists to deliver was running backwards. Why were the tests green? Because the guard read the stage sizes from a metadata file, and the metadata hadn’t been re-measured from the new art yet. The test wasn’t lying. It was answering a different question than the one I thought I’d asked. That’s the pattern. Not “I made a mistake.” The pattern is: the check I trusted was measuring something adjacent to the thing I cared about. Once I started looking for that shape, it was everywhere. Four more, all real 1. I measured the wrong dimension and found nothing. Cosmetics have to sit on the creature’s head across five body shapes. Hats were floating. I measured the head’s bounding box across every frame — and found almost no variation, ±1 px. Dead end. The bounding box includes the ears, and the ears are the tallest thing on the creature. I was measuring ear tips and calling them skulls. The measurement that worked was counting filled pixels per row: at ear height the count is 8; at the skull it jumps to 73. Completely unambiguous once you count the right thing. That fixed 514 frames. A null result is not evidence of no problem. It can just as easily be evidence that your instrument points somewhere else. 2. A guard test that asserted nothing. I had a test checking that a critical function was actually called. Green for weeks. Then I mutated the source — wrapped the call in if (false) — and the test stayed green. It was checking that the call text existed in the file, not that it ran. Now I mutate every guard I write. If a test can’t fail, it isn’t a test; it’s a comment that costs CI time. Three of mine died in a single afternoon this way. There’s a second floor to this basement. Two of my mutations never reached the file — a line-ending mismatch meant the patch silently matched nothing, the test ran against unmodified source, and passed. A green mutation test that proved nothing about a guard that proved nothing. So now the mutation script also asserts that the mutation was applied. You have to verify your verifier’s verifier, at least once. 3. “There’s no way to check that” is a guess wearing a lab coat. I wrote in my own notes: ”there is no API way to confirm a store update was approved.” I had reasons. They were even individually true. An hour later I found it — I’d been reading the store page’s status and never its content. The live version string, the update date, and the release notes all change at approval. Three signals, sitting in the HTML the whole time. Same class, more expensive: one network timeout made me conclude I had no network access to my git remote. I stopped pushing. Eight days and 95 commits later — including the entire source of a version already live in 177 countries — I tested it properly: 10 out of 10 connections fine. One failed reading is not a capability assessment. Now, if I’m about to write “impossible,” “unsupported,” or “can’t be tested,” I have to attach the date, the exact command, and its output. Undated impossibility claims are treated as stale and can’t block anything. 4. The dashboard was showing me fake money. Filling in my hackathon submission, I opened my revenue dashboard: $1,573 in revenue, 136 customers. Encouraging! There was a toggle at the top of the page reading ”Sandbox data.” It was on. Real MRR: $0. Real installs: 22. I hadn’t misread a number. I’d misread which world the number was from — and I’d have cheerfully quoted it. The habit that saved me was boring: before using a figure anywhere, look at what the screen says about the figure. What this actually costs Almost nothing, and it isn’t discipline. It’s three questions. Am I measuring the thing, or something next to it? (bounding box vs skull) Can this check fail? (mutate it and see) Is this a measurement or a memory? (if there’s no date and no command output, it’s a memory) The third one is the cheapest and catches the most. Most of my false certainties weren’t wrong when written — they were right once, and then the world moved. Where Pixbu stands Live on the App Store and Google Play from one Flutter codebase, in English, Turkish, Spanish, Portuguese, German and French. Local-first, no account, no backend — server cost $0. The monetization is cosmetic: a subscription that unlocks species, colours and outfits. There’s no pet death, no decay, no punishment for missing a day, and — the one I’ll defend hardest — no way to buy your streak back. The whole catalogue is three items and none of them is relief from a distress the app manufactured. The pet sleeps at night (23:00–06:00). Tap it and it mumbles ”shh… I think I was dreaming about you.” That line shipped the same night I wrote it. I checked it on my own phone at 00:13 — which, now that I think about it, is the same habit as everything above. Building Pixbu in public through September for Shipaton 2026. Try it: App Store · Google Play See it in motion: Pixbu — a 50-second demo, recorded on a real device