Structured Output Collapses Answer Diversity Across 44 Language Models

Short summary

Asking LLMs to reply in JSON significantly reduces answer diversity across 44 models, with modal answers rising from 41% to 64% on open-ended prompts. The effect is tied to tool-use post-training: JSON and XML compress diversity, while YAML and CSV do not. Decoder-level schema enforcement adds no further compression beyond the text request itself. The finding implies models behave more homogeneously in production structured-output settings than on the chat surfaces where they are evaluated.

• •JSON format requests collapse answer diversity across 44 LLMs, raising modal answer share from 41% to 64%
• •Effect is specific to tool-use-trained formats (JSON, XML) and absent for YAML, CSV
• •Schema enforcement at decoder level adds no further compression — the collapse is in the model's response to register

Generated with AI, which can make mistakes.