What happens when an LLM never sees material beyond fifth grade?

Researchers created LittleLearner, a language model trained exclusively on elementary school curriculum (K-5 level), to study whether models can acquire knowledge beyond their training data through scaling, post-training, or in-context learning. Their findings show that while these interventions improve performance on material within the curriculum, none meaningfully enhance the model's ability to handle advanced concepts outside its training scope, suggesting that pretraining data acts as a hard capability ceiling. The controlled experiment provides a sandbox for cleanly studying how models learn and what determines their knowledge boundaries.