Δ-Mem: Efficient Online Memory for Large Language Models

Researchers propose δ-mem, a lightweight memory mechanism that enables large language models to efficiently store and reuse historical information without expanding context windows or fine-tuning the model. Using only an 8×8 state matrix updated through delta-rule learning, δ-mem achieves 1.10-1.31× performance improvements over baseline models on memory-heavy benchmarks while preserving general capabilities. The approach works by compressing past information into a compact online state that generates low-rank corrections to the model's attention computation during generation.