Structured Output — Make It Return JSON

In one line · When your app needs to read the model's reply in code, you ask for a fixed shape like JSON with specific keys — not free prose.

Glossary:GuardrailsLLMPrompt

Why it matters

Free prose is nice for humans but a nightmare for code. A fixed shape lets you parse the reply and build on it without guessing. Just know: asking for JSON in the prompt doesn't guarantee valid JSON — you still validate, and if your API supports structured output / a schema mode, that's far more reliable than asking in words.

Like the difference between someone telling you their address in a sentence, versus filling a form with fields: street, city, number. The second is machine-readable instantly.

An analogy

Think of it as an API contract with the model: you define the schema, it fills it in. The clearer you specify, the steadier what comes back.

Instead of 'this message is a complaint and its tone is angry', you ask for: {"category":"complaint","sentiment":"angry"} — read straight in code, no regex.

How it works

You describe the shape inside the prompt: the keys, each value's type, the allowed values (enums), ideally plus one full example of the expected reply. The model tries to comply — but it's probabilistic compliance: it may prepend a sentence before the JSON, drop a key, or invent a value outside the enum. So in code you always parse + validate (with a library like Zod), and if your API supports structured output or a JSON schema mode, you use it — it enforces the shape at generation level instead of by request.

Extract from the ticket below. Reply with JSON only, no prose:
{"category": "bug" | "billing" | "other",
 "sentiment": "angry" | "neutral" | "happy",
 "summary": string}

Ticket:
"""
I was charged twice this month and nobody replies to my emails!
"""

Even with a prompt this explicit, treat the reply as untrusted input: parse, then validate, before building on it.

Practice

🧪 Try it yourself: take 5 real customer messages and ask the model to classify them with the schema above. Parse the replies in code and record how many fail, then add one full example output inside the prompt and rerun — compare the valid-JSON rate before and after.

Common mistakes

Calling JSON.parse on the reply directly with no validation and no handling of stray text around the JSON — the first malformed reply throws in production.
A vague schema ('return JSON with the details') without concrete keys and types.
Ignoring the API's supported schema mode and relying on asking in words — leaving your most reliable tool on the shelf.

When to use it

Any reply read by code: classification, extraction, filtering, any pipeline where the model is a step in the middle.

When NOT to use it

Replies going straight to a human (chat, explanations, writing) — JSON there worsens the experience and needlessly constrains the prose itself.

LET'S BUILD ITDone reading — now build it yourself.Make the model return valid JSON

Test your understanding

1. You wrote 'return valid JSON only' in the prompt. Is that a guarantee of valid JSON?

2. Parsing occasionally breaks in production even though the prompt asks for JSON. Strongest fix?

3. When is structured output the wrong choice?

Related concepts

Few-Shot — Teach by Example Anatomy of a Prompt — What It's Made Of Prompt Iteration — Improving It in Passes

Was this clear?