Stochastic Sampling is Epistemically Shallow: The Dimensionality Gap Between Temperature Variation and Model Diversity in LLMs

Short summary

This paper shows that temperature-based sampling variation within a single LLM provides accurate per-question uncertainty but no meaningful cross-question structure. Using a Marchenko-Pastur random-matrix test, the authors find that running one model 100 times at temperature 1 yields at most one signal dimension, while a diverse ensemble of 24 models surfaces four. Only true model diversity, not stochastic sampling, reveals what a model does not know.

• •Temperature variation in a single LLM gives per-question uncertainty but no cross-question structure
• •Marchenko-Pastur test shows at most one signal dimension from sampling vs four from diverse ensembles
• •Self-consistency is epistemically shallow; only ensemble diversity surfaces systematic knowledge gaps

Generated with AI, which can make mistakes.