The original title is: "I type-check AI-generated SDK code against the real package. Claude refused a third of my Stripe tasks."

Short summary

The author built SDKProof, a tool that type-checks AI-generated SDK code against real installed packages using tsc --noEmit. When testing Claude against Stripe tasks, they discovered a harness bug where empty model outputs (refusals) were silently counted as passes. After fixing, rigorous testing revealed Claude refuses 41.3% of Stripe tasks — specifically those involving moving money toward you or reading customer data in bulk — while refusing 0% of Zod tasks.

• •Built SDKProof: type-checks AI-generated SDK code against real packages using tsc
• •Discovered harness bug where empty model refusals were counted as passes
• •Claude refuses 41.3% of Stripe tasks (money-in, bulk customer reads) vs 0% for Zod

Generated with AI, which can make mistakes.