Dev.to
7/9/2026

The headline is about a bug where a scorer was dropping half its shot scores. Let me rewrite this concisely.
Original: The Scorer Was Silently Dropping Half Its Shot Scores
Short summary
A vision-QA scorer in the Poindexter project was silently dropping ~42% of shot scores because qwen3-vl's think trace consumed a too-small 300-token budget before producing JSON output. The fix raised the token limit to 1024 and also addressed a hardcoded timeout override and missing module-registration checks that allowed silent production degradation. Two AST-based test guards were added to prevent future modules from silently failing at boot time.
- •Vision scorer dropped ~42% of scores due to insufficient token budget for qwen3-vl think trace
- •Fixed hardcoded timeout=300 that overrode a correctly-sized 900s client timeout
- •Added AST-based completeness tests to catch unregistered module imports and Dockerfile copy gaps
Generated with AI, which can make mistakes.
Is this a good recommendation for you?



