Matching frontier LLMs at 22 lower latency: a 184M-parameter intent classifier for healthcare text

Short summary

A developer built a 184M-parameter DeBERTa fine-tune for healthcare patient intake classification that matches frontier LLM accuracy (95%+) while running 22× faster on CPU (48ms vs 1000+ms API latency) and costing effectively zero per inference versus $0.25–$0.76 per 1K for Claude/GPT-4o. The key insight: synthetic training data must include realistic messiness—typos, fragments, lowercase—to avoid overfitting to polished LLM-generated examples. Model open-sourced on HuggingFace; part of ClarioScope SLM Suite.

• •184M fine-tuned DeBERTa matches frontier LLM accuracy (95%+) on healthcare intake classification
• •22× faster latency (48ms CPU vs 1000+ms API), ~$0 per inference vs $0.25–$0.76/1K for frontier models
• •Synthetic training data must include realistic mess (typos, fragments) to generalize to real patient text

Generated with AI, which can make mistakes.