16 Days, 4.7M Params, Zero Black Boxes: Building a White-box Chinese Cognition Engine from Scratch

Short summary

Wei Jinqi built a 4.7M-parameter Chinese language model in 16 days with full interpretability—every weight and decision is traceable, unlike black-box LLMs. The design chains specialized modules with learnable gates, debugged through 7 major bugs including mode collapse and gradient breaks. At 92.4% accuracy, it trades performance for complete transparency, proving explainable AI is achievable at modest scale.

• •Built a fully interpretable Chinese language model (4.7M params) in 16 days by designing specialized modules instead of end-to-end training
• •Documented 7 major bugs (mode collapse, gradient chain breaks, repetition collapse) and their fixes, enabling complete model transparency
• •Achieved 92.4% word accuracy with 141MB GPU memory, proving the interpretability-performance tradeoff works at modest scale

Generated with AI, which can make mistakes.