A CSV sample that looks like the whole dataset is worse than no CSV at all

Short summary

The author of data2prompt, a tool that converts data-heavy projects into LLM-readable files, shares a key design lesson: sampled CSV data without explicit context notices causes LLMs to hallucinate statistics from tiny samples. The solution is a consistent annotation grammar (-- [Sample: random 15 of 1,234,567 rows] --) that travels with every reduced dataset, taught once in the system prompt. The article also details real bugs surfaced while auditing against this output contract, including silent Excel chart detection failures.

• •Sampled data without explicit count notices causes LLMs to hallucinate statistics from tiny subsets
• •Solution: consistent -- [Category: detail] -- annotation grammar taught in system prompt, applied to all reductions
• •Auditing against the output contract surfaced real bugs including silent Excel chart detection failure in openpyxl read-only mode

Generated with AI, which can make mistakes.