The SDK You Pick Matters More Than the Model — A 13-LLM Benchmark on the Same Agentic Task

If you have ever built an agent that walks a codebase, calls tools, and writes structured output, you have hit the same wall I kept hitting: the same model produces wildly different results on the same task depending on what harness you wrap it in. Swap Claude for GPT behind a single OPENAI_BASE_URL and you lose half your output quality. Everyone blames the model. The model is rarely the variable. I ran an experiment to put a number on it. Thirteen LLMs — Claude Opus 4.7, Sonnet 4.6, Haiku 4.5,...