Human-Like Neural Nets by Catapulting

A researcher proposes that human-like artificial neural networks could be created by training massively overparameterized models at high learning rates on small, diverse datasets—a technique called "catapulting" that would trigger sudden generalization similar to the grokking phenomenon observed in neural networks. This approach would theoretically produce AI systems with better generalization, adversarial robustness, and sample efficiency by minimizing bias rather than variance, similar to how biological brains learn. The proposal suggests testing this hypothesis by training very large models for fewer steps with cyclical learning rate schedules and benchmarking performance on tasks like arithmetic and image classification.