What Breaks When Your App Has to Reason About Chinese Characters

Short summary

Building software that reasons about individual Chinese characters breaks assumptions at every layer: JavaScript UTF-16 surrogate pairs inflate length, MySQL utf8 truncates Extension B characters, NFKC normalization destroys fullwidth signal, and simplified-to-traditional mapping is not bijective. The author shares hard-won rules: use utf8mb4_bin, NFC only, OpenCC for phrase-level conversion, and store both scripts explicitly rather than auto-converting.

• •JS .length counts surrogate pairs as two — use spread/for...of for CJK character iteration
• •MySQL utf8 is 3-byte and silently truncates Extension B characters — use utf8mb4_bin for exact lookups
• •Simplified-to-traditional Chinese is not a bijection; store both scripts and let users choose rather than auto-converting

Generated with AI, which can make mistakes.