Stop Engineering Prompts: How an Eval-First Harness Let Us Ship 25 Algorithm Versions Autonomously

tl;dr — Agents are good at small fixes and terrible at "make this algorithm better" because every change looks good in isolation and silently regresses elsewhere. We built an AI harness — immutable test set, multi-axis rubric, sweep tool, independent reviewer agent, human-viewable eval interface, knowledge-persistence layer — that lets an agent iterate on a real algorithm autonomously while a human still contributes intuition at the right layer. Twenty-five shipped versions of our color quantiza...