A Classifier That Teaches Itself: Self-Improving, Frozen-gate Training (SIFT) for Dynamic Document Classification

Short summary

SIFT is a self-improving document classifier that pairs a cheap SPLADE+LightGBM pipeline with an LLM judge for low-confidence cases. The judge's labels feed back into the training corpus, so accuracy compounds with use while escalation costs fall. A frozen-gate promotion mechanism with F1 regression checks prevents silent model degradation, making autonomous retraining safe.

• •SIFT pairs cheap SPLADE+LightGBM classifier with LLM judge for low-confidence escalations
• •Judge labels feed back into corpus, reducing escalation rate and growing accuracy over time
• •Frozen-gate promotion mechanism with F1 regression checks prevents silent model regression

Generated with AI, which can make mistakes.