On the Depth Scalability of Logic Gate Networks

Short summary

This paper identifies why Logic Gate Networks (LGNs) fail to benefit from increased depth: optimization collapse and topology-induced information limitations. The authors introduce Input-Anchored LGNs (IALGNs), where each gate combines hidden features with direct input access, enabling consistent depth-accuracy improvements beyond 100 layers on MNIST, CIFAR-10, and CIFAR-100. The work demonstrates that scalable depth requires both stable optimization and an information-access pattern supporting input-conditioned refinement.

• •Two causes identified for LGN depth failure: optimization collapse and topology-induced information limitation
• •Input-Anchored LGNs (IALGNs) preserve a computational spine while conditioning every layer on original input
• •Consistent depth-accuracy improvements beyond 100 layers on MNIST, CIFAR-10, CIFAR-100 where alternatives saturate or degrade

Generated with AI, which can make mistakes.