LRU is harder to beat than the KV-cache papers suggest

A researcher replayed real LLM agent sessions through a prefix-cache simulator and found that LRU (Least Recently Used) eviction policy is harder to beat than recent academic papers suggest, failing three different optimization attempts. The analysis reveals that under capacity pressure, most cache misses come from tool-calling loops seconds apart rather than idle sessions, and the commonly-assumed time-to-live eviction mechanism never even triggered. This challenges the growing literature arguing LRU is suboptimal for agentic LLM workloads and suggests the actual waste patterns differ significantly from what previous research assumed.