AI Evals, Part 4: LLM-as-Judge, Done Right

Short summary

Using an LLM as a judge to score AI outputs requires careful rubric design, defensive JSON parsing, and strategic model selection — specifically, use a stronger, independent model to judge than to generate. LLM judges exhibit systematic biases (position, verbosity, self-preference, sycophancy, and scale compression) that must be deliberately mitigated through anchored dimension descriptions and order randomization. Always validate judge agreement against human labelling using Cohen's κ on a held-out set before deploying to production; a κ ≥ 0.6 is usable.

• •Use a stronger model to judge outputs than to generate them; treat evaluation as a production AI feature
• •LLM judges exhibit position bias, verbosity bias, self-preference, sycophancy, and scale compression — each requires specific mitigations
• •Validate against human labels using Cohen's κ (≥0.6 is usable) before trusting production scores; retune rubrics on train/test split

Generated with AI, which can make mistakes.