Lesson 04 – Understanding `response_format={'type': 'json_object'}` in LLM APIs¶

When you interact with Large Language Models (LLMs) like OpenAI's GPT models (e.g., GPT-4o, GPT-3.5-turbo), you often want the output in a specific, machine-readable format, such as JSON. The response_format parameter in the API call is designed to help the model understand and adhere to this requirement.

The most common setting for this parameter is {"type": "json_object"}.

What `response_format={'type': 'json_object'}` Means¶

When you set response_format={'type': 'json_object'} in your API request, you are explicitly instructing the LLM to generate a single, top-level JSON object.

{
  "key1": "value1",
  "key2": "value2"
}

Why This Impacts Retrieving Multiple Items¶

A "JSON object" is not a JSON array. If your prompt asks for an array but the API parameter enforces a single object, the LLM will prioritize producing that single object. The model might attempt to wrap your array inside an object or return only the first item, sometimes resulting in malformed JSON.

Aligning Instructions¶

To reliably get a JSON array containing multiple items, ensure your prompt and API parameters do not conflict.

Method 1: Removing `response_format`¶

For modern models like gpt-4o, omit the parameter and rely on a strong prompt instructing the model to output a JSON array.

response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    temperature=0.2,
    max_tokens=2000,
    # response_format omitted
)

Method 2: Wrapping an Array Inside a JSON Object¶

If you must use response_format={'type': 'json_object'}, modify the prompt so the array is under a single key within that object.

response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=messages,
    temperature=0.2,
    max_tokens=2000,
    response_format={"type": "json_object"}
)

Prompt:

Output a JSON object with a key `action_items` whose value is a JSON array...

Then access response_data['action_items'] in your code.

In Conclusion¶

response_format={'type': 'json_object'} ensures a valid single JSON object. When you need a list of items, either omit the parameter and instruct the model via the prompt, or wrap the array inside a named key within the object.

Understanding how response_format={'type': 'json_object'} shapes model output helps you retrieve predictable data structures. Building on Lesson 03 – Using the OpenAI API, which emphasized reliable programmatic interaction, you'll next explore Lesson 05 – Using ChatGPT Codex for Development to accelerate coding tasks with conversational assistance.

Lesson 04 – Understanding response_format={'type': 'json_object'} in LLM APIs¶

What response_format={'type': 'json_object'} Means¶