I scaled a pure Spiking Neural Network (SNN) to 1.088B parameters from scratch. Ran out of budget, but here is what I found

Short summary

An 18-year-old indie developer successfully trained a 1.088B-parameter Spiking Neural Network purely from random initialization, achieving 93% sparsity and converging loss to 4.4 before running out of budget at 27k steps—proving direct SNN training is feasible despite conventional wisdom. The model spontaneously developed cross-lingual capabilities (structurally correct Russian without explicit training) and shifted 39% of activation routing into persistent memory as it scaled from 600M to 1B parameters. Full code, architecture details, and the reproducible 12GB checkpoint are open-sourced on GitHub.

• •Pure SNN training from scratch succeeded where papers claim it fails, reaching 1.088B parameters with 93% sparsity despite budget constraints
• •Model emerged with cross-lingual capabilities and adaptive memory routing without explicit training signals or multilingual dataset weighting
• •Reproducible artifact (code + full 12GB checkpoint) open-sourced; author requests technical feedback on surrogate gradients and neuromorphic hardware mapping

Generated with AI, which can make mistakes.