Structured outputs

When code consumes a model's answer, you need data rather than prose: a classification, extracted fields, a plan your program can run. There are three ways to get it, from least to most reliable.

1. Ask for JSON in the prompt. It usually works, but sometimes you get a trailing comment, a missing field or a wrapper sentence.
2. Use a tool definition as the schema. Define a tool whose input schema is the shape you want and require the model to call it. The arguments come back as structured JSON. See the tool-use loop.
3. Use structured outputs, or constrained decoding. You pass a JSON Schema and the provider restricts token sampling so the output can only be valid against it. Check which schema features your provider supports.

from typing import Literal
from pydantic import BaseModel

class Ticket(BaseModel):
    category: Literal["billing", "bug", "feature_request", "other"]
    urgency: int            # 1-5
    summary: str

# Pass Ticket's JSON Schema to the API's structured-output option, then:
ticket = Ticket.model_validate_json(response_text)   # still validate on your side

When the whole answer is a label, a level or a yes/no, a decision model is an alternative: it returns a probability for every allowed value instead of one generated choice.

#Valid isn't correct

A schema guarantees shape, not truth. The model can still choose the wrong category, invent a value it couldn't find, or put 5 for urgency on every ticket. So:

• Validate semantically after parsing: check ranges, check that referenced IDs exist, check cross-field rules.
• Give the model a way out. Add null, "unknown" or an "other" enum value so it isn't forced to make something up when the answer isn't there.
• Put a reasoning field first when the task needs thought, for example {"reasoning": "...", "category": "..."}. Fields are generated in order, so the model reasons before it commits.
• Keep schemas small and flat. Deeply nested optional structures are harder for the model to fill well.

#Habits

• Use one schema definition for everything. Generate the JSON Schema from your Pydantic model or Zod schema, so the prompt, the API call and the validation can't drift apart.
• Describe the fields. Schema description fields are part of the prompt, and good descriptions noticeably improve accuracy.
• On failure, retry with the error. Send the validation message back once and ask for a correction, then fail loudly.
• Measure field accuracy with evals, not just parse success.

In TypeScript, a Zod parse gives you a properly narrowed type for the rest of your code.