What an LLM Actually Does: Predicting the Next Word, Explained

Short summary

LLMs predict the next word from a probability distribution, then repeat with updated context. Temperature reshapes these probabilities—low values yield safe repetition, high values enable creativity. Hallucinations occur because models optimize for likelihood, not correctness.

• •LLMs operate via a predict-append-repeat loop over probability distributions across 100K+ vocabulary
• •Temperature is the single control for "creativity": low = deterministic/safe, high = creative/error-prone
• •Hallucinations result from optimizing for plausible-sounding words rather than factual accuracy

Generated with AI, which can make mistakes.