API Design for AI: Streaming, Structured Output, and Rate Limits

API Design for AI: Streaming, Structured Output, and Rate Limits Most API design guides treat endpoints like they're serving JSON from a database. But Claude and other LLMs break that assumption: responses take 5–30 seconds, tokens cost money, and users expect real-time feedback. I've shipped CitizenApp with nine AI features across three product tiers, and I've learned the hard way that standard REST patterns don't cut it. This post covers three patterns I now use by default: streaming respon...