Do Transformers Need Three Projections? Systematic Study of QKV Variants

Researchers systematically evaluated whether transformer models need three separate query, key, and value (QKV) projections in attention mechanisms, finding that sharing projections—particularly combining query-key and value (Q-K=V)—performs comparably to standard QKV while reducing KV cache by 50% with minimal performance loss. When combined with existing optimization techniques like grouped query attention, projection sharing achieves up to 96.9% cache reduction, enabling practical deployment on edge devices with limited memory.