An AI Feature Has No "Tests Pass" Moment. So I Write the Eval First.

Short summary

Write evaluations before building AI features—unlike traditional software, AI outputs look plausible when wrong, so evals define success rather than verify existing implementations. The author's spoiler-detection example shows how a hard eval requirement forced better architecture (retrieval gates vs. just prompts). With measurable criteria, teams build systems that work, not guesses that merely demo well.

• •Eval-first development is TDD for AI—success criteria drive architecture, not the reverse
• •Prompts alone can't guarantee hard requirements like zero spoilers; retrieval gates add the missing safety layer
• •Without measurable evals, AI features silently ship with plausible-looking but incorrect outputs

Generated with AI, which can make mistakes.