[R] All Routes Lead to Collapse: attention sinks, representation collapse, and norm stratification are what content-based routing does under a norm-blind metric

Short summary

Research paper argues that transformer pathologies (attention sinks, representation collapse, norm stratification) stem from norm-blind routing metrics, not attention per se. Authors demonstrate this across five architectures with causal ablations and release code.

• •Attention pathologies trace to norm-blind content-based routing, not attention mechanisms
• •Demonstrated across transformers, GAT, Mamba, RWKV, and AttnRes with ablation studies
• •Code and data available; shows positional priors determine onset depth, not norm stratification

Generated with AI, which can make mistakes.