Fine-tuned LLMs classify vulnerability in 3,000 UK police logs

Short summary

Researchers fine-tuned an open-weight LLM to classify vulnerability indicators (mental ill health, substance misuse, alcohol dependence, homelessness) across ~3,000 UK police incident logs. Naive single-pass classification was unreliable and over-assigned indicators, requiring substantial human review and statistical correction. Population-level estimates were achievable but resource-intensive; individual-level errors remained too frequent for operational decisions.

• •LLM pipeline classifies four vulnerability indicators in UK police incident logs using local open-weight model
• •Single-pass classifications are unstable and systematically over-assign indicators vs human judgment
• •Defensible population-level estimates achievable with heavy human input, but individual-level errors limit operational use

Generated with AI, which can make mistakes.