What is AI Structured Output? How JSON Schema makes LLM JSON reliable

Why prompt-only JSON is unreliable

“Return JSON only, no explanation” is the usual workaround. It sometimes works, because it writes a preference into context. It is unreliable, because a preference is not a constraint: the model can still wrap the object in a markdown fence, drop a required key, or turn priority into the more “natural” urgency. Once downstream code hits JSON.parse, the failure is not a copy issue — the whole pipeline stops.

The quieter failure is “it parses, but the shape is wrong”. You expected items to be an array and got an object; you expected a numeric score and got "0.91". The code reads undefined or concatenates strings, and the bug surfaces much later. A prompt cannot stop that drift, because it never rejects illegal tokens at decode time.

So do not treat Structured Output as “a stricter prompt”. It is part of generation: the server compiles the schema into the next allowed token set, and the model cannot emit unmatched brackets or a key that is not on the list. The prompt still explains the task; the schema owns the shape.

What layer Structured Output actually constrains

Think of three layers. Mix any two and it feels like “the model is being random”. Layer one is syntax: the output must parse as JSON. Layer two is shape: names, types, required fields, and enums must match the schema. Layer three is meaning: did it pick the right class, is the amount real? Structured Output covers the first two. The third is always yours.

Tier	What you configure	What you actually get
Prompt convention	“JSON only”	A preference, not a contract
JSON mode	MIME type or `json_object`	Likely valid JSON; field names still belong to the model
Schema mode	JSON Schema plus a strict flag	Shape, types, required fields, and enums follow the schema

OpenAI describes Structured Outputs as the next step after JSON mode: both can produce valid JSON; only the former guarantees the schema you supplied. Official comparison: Structured model outputs. Gemini draws the same line between “just JSON” and “emit fields from a schema”; for the switches, see the Gemini API JSON output guide — this article will not repeat the SDK details.

One easy-to-miss boundary: a safety refusal, a truncation, or a failed tool call may not fit your success object. Some APIs return a separate refusal or empty content. The schema constrains the stretch that “speaks in the format”, not “this call will succeed”.

How JSON Schema becomes a contract

JSON Schema is a vocabulary for JSON documents: types, required fields, enums, numeric ranges, array items. The standard walkthrough is Understanding JSON Schema. Hooked up to a model, that vocabulary gains a second job: it is no longer only a post-check — it narrows the search space while tokens are generated.

For engineering, the schema is a contract shared by compile time and runtime. Pydantic, Zod, and Swift generable types usually collapse to one language-agnostic JSON Schema so you can hand it to a cloud model, log it, and replay fixtures. Keep one field table so the app does not say totalCents, the API amount, and the prompt “amount”.

Write the object you will actually read, not a complete world model. Every extra optional field is another chance to fill it wrong or leave it blank. Required fields go in required, closed sets in enum, numeric bounds in minimum / maximum. A property description is usually more stable than repeating the explanation in the prompt, because the constraint is bound to decoding.

Strict mode usually adds one more rule: objects must set additionalProperties: false, and every declared field goes in required. For a truly optional value, do not “omit it from required” — allow null. Otherwise the model may drop the key while your code still assumes obj.field always exists.

How each API attaches a schema

Vendors all say Structured Output; the wrapper fields differ. Ask two questions first: does this schema constrain the final answer or the tool arguments? Does this model snapshot actually support strict mode? Do not assume a keyword in the docs behaves the same on every endpoint.

OpenAI: json_schema plus strict

In Chat Completions, set response_format to json_schema and turn on strict: true. The Responses API writes the same thing under a text-format field. The schema rules match; only the wrapper changes. The request below extracts a ticket: category is an enum, account may be null.

{
  "model": "gpt-4o-2024-08-06",
  "messages": [
    { "role": "system", "content": "Extract the ticket into the schema." },
    { "role": "user", "content": "Checkout 500s on a saved card. Account acct_8842." }
  ],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "support_ticket",
      "strict": true,
      "schema": {
        "type": "object",
        "properties": {
          "summary": { "type": "string" },
          "category": {
            "type": "string",
            "enum": ["billing", "bug", "account", "other"]
          },
          "severity": { "type": "integer" },
          "account_id": {
            "anyOf": [{ "type": "string" }, { "type": "null" }]
          }
        },
        "required": ["summary", "category", "severity", "account_id"],
        "additionalProperties": false
      }
    }
  }
}

Treat the response as “the whole thing is JSON”. Do not regex a code fence first. The SDK parse helper can deserialize into a typed object; also handle the refusal branch — a safety trip may not match the success schema.

Gemini and the other stacks

Gemini declares “this is JSON” with a MIME type, then pins fields with responseSchema or responseJsonSchema. The latter is closer to standard JSON Schema and fits anyOf, $ref, and numeric ranges. Details and Python / JS samples: the Gemini Structured Output tutorial.

Apple’s on-device models use @Generable — a compile-time shape, not a JSON Schema file you typed by hand. Once you leave the process, hit HTTP, or write a log, you still need serializable JSON. How the three call chains split: Apple AI agents and JSON. When an external agent uses MCP or REST, the payload is almost always JSON, and the schema is still the table the adapter should align.

A schema you can ship

The schema below models ticket classification: a closed category, an integer priority, a string summary. That is what a schema should own — shape, not “should this ticket be urgent in the business sense”.

{
  "type": "object",
  "properties": {
    "category": {
      "type": "string",
      "enum": ["billing", "bug", "feature"],
      "description": "Ticket category"
    },
    "priority": {
      "type": "integer",
      "minimum": 1,
      "maximum": 5
    },
    "summary": {
      "type": "string"
    }
  },
  "required": ["category", "priority", "summary"],
  "additionalProperties": false
}

Describe array elements with items. For a fixed-length, tuple-like list, look at prefixItems on the JSON Schema path. Inline nested objects first. Reach for $defs + $ref only when the same structure appears a third time, or a tree node points at itself. Abstract too early and errors get harder to read: when the server rejects the schema, you are staring at one expanded blob.

Do not describe the structure once in the prompt and again in the schema. Duplicate descriptions make the model oscillate between two stories and waste input tokens. Keep the task in the prompt; keep names and types only in the schema. When local types change a field, the request schema must change with them, or production silently grows extra keys or missing ones.

You still validate locally after the constraint

Structured Output quiets the parse layer. It does not guarantee the class is right, or that a number respects your business cap. An enum limits the set; it cannot stop a wrong pick. Priority 5 is valid; “it should have been 2” is also valid JSON. Sample the critical path or add rules.

In production, treat the model output as ordinary JSON. JSON formatter to see the nesting, JSON validator to confirm it parses, then drop the same object into JSON Schema for a second local check. Against a gold set, use JSON Diff to spot key-order or nullable drift in one glance.

Keep three fixtures: ticket.valid.json, ticket.missing-field.json, ticket.wrong-enum.json. The first confirms the happy path; the next two confirm your local check actually rejects. Errors the model side already constrains should still be reproducible locally — otherwise the day you swap models or turn strict mode off, the pipeline finds out while it is offline.

FAQ

How is Structured Output different from JSON mode?

JSON mode guarantees parseable JSON. Structured Output also guarantees the JSON Schema you supplied: names, types, required fields, and enums follow the contract. If your code reads named fields, use the latter.

If I have a schema, do I still write “JSON only”?

A short line is fine. Do not paste the field table into the prompt. Shape is owned by the schema. Two descriptions lower quality and waste quota.

How do I represent optional fields in strict mode?

Most strict implementations close extra properties and require every listed field. Make optional values nullable — a string-or-null union — instead of deleting the key from required.

Can the business still be wrong if the schema passed?

Yes. A schema owns shape, not truth. A wrong class, a hallucinated amount, or a forced fill instead of a refusal can all be valid JSON. Production still needs rules, sampling, or a human check.

Summary and next steps

Structured Output is not a copy trick. It turns JSON shape from a wish into a decode constraint. JSON Schema is the most common way to write that constraint: required fields, enums, ranges, and no extra keys decide whether downstream can stably JSON.parse and read the fields it expects. Switch names differ by API; the layers do not — syntax, shape, meaning. Do not mix them.

Next, write a small schema you will actually read, run one extract or classify path in strict mode, then validate the return value in the browser. Open the previous guide for Gemini request samples; open the Apple piece to see how agent arguments land as JSON.