Stop Begging Your LLM for Valid JSON: Self-Correcting Structured Output in Spring AI 2.0

Every developer who has worked with LLMs has been there. You ask the model for JSON. You describe the schema. You say "please only respond with valid JSON." And sometimes, it still breaks. Your application crashes because the model returned a string where you expected an integer. Or it wrapped the JSON in markdown code blocks. Or it omitted a required field. Spring AI 2.0 has a solution that treats this like a real engineering problem instead of a prayer. The Problem When you use structured output in Spring AI, the workflow goes like this: You define a Java type (a record, class, or enum) Spring AI generates a JSON schema from that type The schema gets appended to the prompt sent to the LLM The model returns a response Spring AI attempts to deserialize the response into your type This works well with frontier models like Claude and GPT-4. But smaller open-source models, like Llama 3.2 1B running locally via Ollama, fail more often. They might return null for a primitive field, omit required fields, or produce malformed JSON. When it fails, you get a deserialization exception. Your endpoint returns a 500 error. Spring AI provides no built-in recovery mechanism. The Old Approach: Hope Consider a conference talk submission system. Speakers submit messy, unstructured abstracts. You want to extract structured data: public record TalkSubmission( String title, String abstractText, Level level, // BEGINNER, INTERMEDIATE, ADVANCED Track track, int duration, List tags, String speakerHandle ) {} Enter fullscreen mode Exit fullscreen mode Here is what the basic typed response looks like: @PostMapping("/typed") public TalkSubmission typed(@RequestBody String rawSubmission) { return chatClient.prompt() .system(systemPrompt) .user(spec -> spec.text("Extract the talk submission: {submission}") .param("submission", rawSubmission)) .call() .entity(TalkSubmission.class); } Enter fullscreen mode Exit fullscreen mode You define your type. Spring AI generates the schema and appends it to the prompt. The model gets the instruction. And you hope it works. Dan Vega, Spring Developer Advocate at Broadcom, puts it bluntly in his video demonstration: "That's not engineering. That's hoping." The New Solution: validateSchema() Spring AI 2.0 introduces self-correcting schema validation. When the model returns invalid JSON, Spring AI validates the response against the schema, detects the error, feeds the error message back to the model, and asks it to fix the response. The change is a single method call: @PostMapping("/validated") public TalkSubmission validated(@RequestBody String rawSubmission) { return chatClient.prompt() .system(systemPrompt) .user(spec -> spec.text("Extract the talk submission: {submission}") .param("submission", rawSubmission)) .call() .entity(TalkSubmission.class, spec -> spec.validateSchema()); } Enter fullscreen mode Exit fullscreen mode That is it. The spec -> spec.validateSchema() consumer turns on the self-correcting retry loop. How It Actually Works According to the official documentation by Christian Tzolov (Spring AI team), the validation loop works as follows: The model responds Spring AI validates the response against the generated schema If validation passes, you get your typed record back If validation fails, the specific validation error (e.g., "expected int, got null for field duration") is appended to the user prompt and the call is re-issued The model sees the exact error on each retry, not a blind re-try This is powered by StructuredOutputValidationAdvisor, a recursive advisor that is auto-registered when you call validateSchema(). Default is 3 retry attempts. The model knows exactly what went wrong and can correct it on the next attempt. To customize the retry count, build your own advisor instance: var validationAdvisor = StructuredOutputValidationAdvisor.builder() .outputType(TalkSubmission.class) .maxRepeatAttempts(5) .build(); ChatClient chatClient = ChatClient.builder(chatModel) .defaultAdvisors(validationAdvisor) .build(); Enter fullscreen mode Exit fullscreen mode Provider-Native Structured Output Some frontier models support structured output at the API level. Instead of appending the schema to the prompt text, the schema is sent as an API constraint. The provider's runtime enforces conformance, meaning invalid responses cannot be emitted at all. Spring AI 2.0 exposes this through useProviderStructuredOutput(): TalkSubmission result = chatClient.prompt() .system(systemPrompt) .user(spec -> spec.text("Extract the talk submission: {submission}") .param("submission", rawSubmission)) .call() .entity(TalkSubmission.class, spec -> spec .useProviderStructuredOutput() .validateSchema()); Enter fullscreen mode Exit fullscreen mode Supported providers as of Spring AI 2.0: OpenAI: GPT-4o and later models with JSON Schema support Anthropic: Claude 3.5 Sonnet and later models Google GenAI: Gemini 1.5 Pro and later models Mistral AI: Mistral Small and later models with JSON Schema support Ollama: Models with JSON Schema support (model-specific) Native structured output is off by default because support varies across models. If a model does not support it, the flag is silently ignored and the prompt-based approach is used instead. Note: .entity() is only available on .call(), not on .stream(). Typed parsing requires the complete response, so streaming responses cannot be deserialized into a typed object. Known Limitations OpenAI does not accept top-level JSON arrays. If you need a List, wrap it in a container record first: // Does NOT work with OpenAI native structured output: List list = chatClient.prompt() .call() .entity(new ParameterizedTypeReference>() {}, spec -> spec.useProviderStructuredOutput()); // fails // Works: wrap in a container record SubmissionList(List submissions) {} SubmissionList result = chatClient.prompt() .call() .entity(SubmissionList.class, spec -> spec.useProviderStructuredOutput()); Enter fullscreen mode Exit fullscreen mode Ollama with reasoning models (like Qwen variants) may emit internal reasoning traces as plain text instead of JSON. Use a non-reasoning model, or combine with validateSchema() so malformed responses are automatically retried. When To Use What Not every scenario needs all features enabled. Based on the official docs and video demonstration: Frontier models (Claude, GPT-4, Gemini): useProviderStructuredOutput() for API-level enforcement validateSchema() as a safety net for edge cases These models rarely fail, but the combination gives you two layers of protection Open-source models (Llama, Mistral via Ollama): useProviderStructuredOutput() may have no effect (model-specific) validateSchema() is essential These models fail more often, especially with complex schemas or small parameter counts Production systems: Always enable validation. The overhead is minimal compared to a 500 error Log validation failures to identify which prompts or models need improvement Customize maxRepeatAttempts based on your latency budget The Bigger Picture This feature represents a shift in how we think about LLM integration. For too long, the industry treated unreliable model output as a prompt engineering problem. Write a better prompt. Be more specific. Add examples. Pray harder. Spring AI 2.0 treats it as a systems problem. Validate. Retry. Self-correct. The same principles we apply to any unreliable external service: network calls, database queries, third-party APIs. LLMs are no different. If you are building production applications with LLMs, schema validation is not optional. It is basic engineering. Sources: Self-Correcting Structured Output in Spring AI 2.0 - Dan Vega (YouTube) Self-Correcting Structured Output in Spring AI 2.0 - Christian Tzolov (Spring Blog) Spring AI Structured Output Reference Docs