Compression Is Prediction

The article explores the fundamental connection between data compression and language models, arguing that both are essentially trying to solve the same problem: predicting patterns in data. It explains how compression works by identifying and exploiting redundancy—such as run-length encoding—and introduces the three core components of modern compressors: transforms, models, and entropy coders. The piece suggests that understanding compression provides insight into how large language models function at a deeper level.