A 125M NER model matches a 14B LLM for medical text de-identification at 40x the speed on CPU

Short summary

A 125M parameter NER model (obi/deid_roberta_i2b2) matched a 14B LLM's redaction recall on medical text de-identification while running 40x faster on CPU. The author built localscrub, a local-first cascade combining rules, NER, and optional LLM stages, with a rigorous three-metric eval harness (relaxed recall, strict recall, redaction recall). The NER-only configuration achieved 0.94 redaction recall at 184ms per note on CPU, proving small specialized models can rival large LLMs for structured extraction tasks.

• •A 125M NER model matched a 14B LLM at PHI redaction recall (0.94) while running 40x faster on CPU
• •localscrub uses a cascade: rules for structured identifiers, NER for names/locations, optional LLM for complex cases
• •Three-metric eval (relaxed, strict, redaction recall) caught real bugs including merge errors and gold-standard flaws

Generated with AI, which can make mistakes.