Structured Output and JSON

When your program, not a person, reads the model's answer, you need the answer in a fixed format. JSON is the usual choice.

Why Structured Output?

Imagine Ayesha runs a small institute. Students send messages like "Hi, I'm Ayesha Khan, I want to join web development, evening if possible, my number is 0300-1234567". She wants each message saved as a row in a spreadsheet.

A normal chat reply ("Sure! The student's name is Ayesha...") is hard for code to read. A JSON reply is easy:

Example

json

{"name": "Ayesha Khan", "course": "Web Development", "phone": "0300-1234567", "wants_evening": true}

Common uses of structured output:

Pulling fields out of messages, invoices or CVs
Classifying text (complaint, question, order) into fixed labels
Producing data for charts, forms or databases
Making the model's answer safe to use in the next step of a program

For prompt tips on formats like tables and lists, see Controlling the Output Format. This chapter focuses on the code side.

Three Levels of Reliability

Method	How	Reliability
Ask in the prompt	"Reply with JSON only, with keys name, course..."	Good, but not guaranteed
JSON mode	`response_format={"type": "json_object"}`	Always valid JSON, but keys may vary
JSON schema	`response_format` with a full schema	Valid JSON that follows your schema

Support for JSON mode and JSON schemas differs between providers and models. Check your provider's documentation, and always validate in your code anyway.

Asking for JSON with the API

Example

python

import json, os
from openai import OpenAI

client = OpenAI(base_url=os.environ["AI_BASE_URL"], api_key=os.environ["AI_API_KEY"])

message = ("Hi, I'm Ayesha Khan, I want to join web development, "
           "evening if possible, my number is 0300-1234567")

response = client.chat.completions.create(
    model=os.environ["AI_MODEL"],
    response_format={"type": "json_object"},
    temperature=0,
    messages=[
        {"role": "system", "content":
            "Extract the student's details. Reply with JSON only, using exactly these keys: "
            "name (string), course (string), phone (string), wants_evening (true or false)."},
        {"role": "user", "content": message},
    ],
)
data = json.loads(response.choices[0].message.content)
print(data)

Two tips: set temperature to 0 for extraction, and when you use JSON mode, also say "JSON" in your prompt. Some providers require it.

If your provider supports JSON schemas, you can describe the exact shape instead:

Example

python

response_format = {
    "type": "json_schema",
    "json_schema": {
        "name": "lead",
        "strict": True,
        "schema": {
            "type": "object",
            "properties": {
                "name": {"type": "string"},
                "course": {"type": "string"},
                "phone": {"type": "string"},
                "wants_evening": {"type": "boolean"},
            },
            "required": ["name", "course", "phone", "wants_evening"],
            "additionalProperties": False,
        },
    },
}
print(list(response_format["json_schema"]["schema"]["properties"]))

Output

['name', 'course', 'phone', 'wants_evening']

Cleaning the Reply

When you only ask in the prompt, models sometimes wrap JSON in a Markdown code fence. A small helper removes it before parsing:

Example

python

import json

# Pretend this text came back from the model
reply = '```json\n{"name": "Ayesha Khan", "course": "Web Development", "phone": "0300-1234567", "wants_evening": true}\n```'

def parse_json_reply(text):
    text = text.strip()
    if text.startswith("```"):
        text = text.split("\n", 1)[1]      # drop the first ``` line
        text = text.rsplit("```", 1)[0]    # drop the last ```
    return json.loads(text)

data = parse_json_reply(reply)
print(data["name"], "|", data["course"], "| evening:", data["wants_evening"])

Output

Ayesha Khan | Web Development | evening: True

Validating with Pydantic

Valid JSON is not enough. A key can be missing, or a value can have the wrong type. Pydantic (pip install pydantic) checks the data against a Python class:

Example

python

from pydantic import BaseModel, ValidationError

class Lead(BaseModel):
    name: str
    course: str
    phone: str
    wants_evening: bool

good = '{"name": "Ali Raza", "course": "Python", "phone": "0321-7654321", "wants_evening": false}'
bad = '{"name": "Ali Raza", "course": "Python", "wants_evening": "maybe"}'

print(Lead.model_validate_json(good))
try:
    Lead.model_validate_json(bad)
except ValidationError as e:
    print("Invalid reply:", e.error_count(), "problems")
    for err in e.errors():
        print(" -", err["loc"][0], ":", err["msg"])

Output

name='Ali Raza' course='Python' phone='0321-7654321' wants_evening=False
Invalid reply: 2 problems
 - phone : Field required
 - wants_evening : Input should be a valid boolean, unable to interpret input

When validation fails, you can send the error back to the model and ask it to fix its answer, try once more, or pass the message to a human.

Valid JSON can still contain wrong facts. If the message has no phone number, a model may invent one. Tell it what to do when a value is missing, for example "use null if not given", and allow None in your class.

Exercise

Your app asks the model for JSON. What should your code do before saving the data?

Nothing, models always return perfect JSONParse it and validate the keys and types, and handle failuresPrint it and hope it is correctConvert it to an image