FALCON-Discover: Discovering Concentrated False-Confidence Regions for Calibration

Short summary

FALCON-Discover is a post-hoc, model-agnostic framework for finding concentrated false-confidence regions in ML predictions—areas where models are confidently wrong. It ranks predictions using discrepancy signals from confidence, local support, neighborhood agreement, and perturbation stability. Across seven tabular datasets with XGBoost and CatBoost, discrepancy-based ranking substantially outperforms standard calibration baselines in detecting dangerous errors, though the best detector varies by dataset regime.

• •FALCON-Discover identifies compact regions where ML models are confidently wrong
• •Uses discrepancy signals from confidence, support, neighborhood agreement, and stability
• •Outperforms standard calibration baselines on tabular datasets with XGBoost and CatBoost

Generated with AI, which can make mistakes.