The One-Step Trap (In AI Research)

Rich Sutton identifies the "one-step trap" as a critical flaw in AI research: the mistaken assumption that learning only one-step predictions and iterating them can generate accurate long-term predictions. While this approach seems intuitive and mirrors physics-based models, it fails in practice because prediction errors compound exponentially, and the computational complexity becomes infeasible in stochastic environments. Sutton argues that the solution lies in developing temporally abstract models using options and general value functions rather than relying on simple one-step transition models.