It's the Memory Latency, Stupid

Modern CPUs can execute 400-800 instructions in the time it takes to access RAM once, making memory latency the primary bottleneck when databases appear CPU-bound at 100%. CPU caches (L1, L2, L3) mitigate this by storing frequently accessed data much closer to the processor, but their small size means poorly designed algorithms—like hash table-based GROUP BY operations with millions of unique keys—fail to utilize these caches and force constant expensive RAM accesses. Effective database performance engineering must prioritize cache utilization by processing data in small chunks that fit within cache hierarchies rather than assuming high CPU utilization indicates efficient processing.