Building Frame-Level Tetris AI from Raw Pixels: Why Training Collapses at 1.4M Gradient Steps

Short summary

A researcher trained an RL agent to play NES Tetris frame-by-frame from raw pixels without handcrafted features or shaped rewards, achieving level 21 before mysteriously collapsing. All four experimental runs with different reward configurations hit a death clock at roughly 1.4M gradient steps. The root cause: credit assignment failure—pieces require tens of frames to place, but the reward signal arrives much later, exceeding the agent's representational capacity.

• •Researcher achieved the first frame-level Tetris AI from raw pixels without handcrafted features, reaching NES level 21
• •All trained agents mysteriously collapsed at ~1.4M gradient steps regardless of reward shaping strategy
• •Root cause: credit assignment—pieces require tens of frames to place but rewards arrive late, exceeding model capacity

Generated with AI, which can make mistakes.