On the Computational Complexity of Structural Generalization

Short summary

This paper formally defines structural generalization and proves that pure Transformers cannot learn it under standard complexity assumptions (TC^0 ≠ NC^1). Each compositional rule splits into syntactic and semantic faces; the semantic face is NC^1-complete while Transformers can only learn TC^0, making the hard half unlearnable. Neuro-symbolic systems score well on benchmarks precisely because they inject the semantic face directly, meaning benchmark scores cannot distinguish learned from given capabilities.

• •Formally defines structural generalization and proves pure Transformers cannot learn it (TC^0 ≠ NC^1)
• •Compositional rules split into syntactic and semantic faces; semantic face is NC^1-complete
• •Neuro-symbolic systems sidestep the hard half by injecting semantic rules, inflating benchmark scores

Generated with AI, which can make mistakes.