The original title is a question about AI agent eval sets. Let me rewrite it to be punchy, specific, and under 12 words.

Short summary

Argues that your AI agent's eval set is more valuable than the prompt it grades because it survives every model swap, prompt rewrite, and tool change. Most eval sets fail because they are written from imagination at the start when you know least about failure modes. The fix: convert every production incident into a permanent test case, weight toward disqualifying failures, include mundane traffic distribution, and use property-based assertions instead of exact string matching for non-deterministic agents.

• •Eval sets are the durable asset; prompts are disposable — invest in curating failure cases
• •Every production incident should become a permanent eval case before the fix is considered done
• •Use property-based assertions (did it refuse unsafe input, call right tool, stay under budget) instead of exact string matching for non-deterministic agents

Generated with AI, which can make mistakes.