When AI Benchmarks Plateau: A Systematic Study of Benchmark Saturation

A systematic study of 60 language model benchmarks found that nearly half exhibit saturation—where models perform so well that the benchmarks can no longer effectively differentiate between them—with saturation rates increasing as benchmarks age. The research identifies expert curation as a key factor in extending benchmark longevity, while public test data accessibility has minimal impact on saturation resistance. The findings suggest that thoughtful design choices can create more durable evaluation approaches for measuring AI progress.