It's the memory latency, stupid
- ID
- 41717
- Status
- summarized
- Published
- 01 Nov 2026, 2:38 AM
- Fetched
- 01 Nov 2026, 10:31 PM
- Provider
- Hacker News
- Category
- dev-community
- Original URL
- https://pivotlake.io/blog/its-the-memory-latency-stupid/
- Source URL
- https://hnrss.org/best
Summary
- Score
- 7.0
- Created
- 01 Nov 2026, 10:31 PM
- Tags
- Audience
- developersdatabase_learners
What happened
A Pivot blog post by Maor Kern argues that a CPU pinned at 100% is often stalled on memory rather than doing useful work: at 4–8 instructions per nanosecond and 50–100ns per RAM access, a modern core could execute roughly 400–800 instructions in the time one DRAM fetch takes, versus ~1ns for L1 (~64KB), ~3–4ns for L2 (~1MB), and ~10–15ns for L3 (up to a few hundred MB). It applies this to GROUP BY aggregation, where the naive approach of a single ever-growing hash table keyed by group means millions of unique groups produce almost all cold, RAM-bound lookups instead of cache hits. The post is framed as making GROUP BY faster by doing more CPU work, though the excerpt cuts off before the specific replacement technique is described. It drew 221 points and 72 comments on Hacker News.
Why it matters
If you're tuning analytics queries or a data pipeline, this gives you a concrete diagnostic: high CPU percent on a GROUP BY with high group cardinality may be memory-stall time, not compute, so adding cores or a faster clock won't help — reducing the hash table's working set (or accepting extra CPU work for cache locality) is the lever. The specific numbers (400–800 instructions per RAM access, ~64KB L1) are the ones to keep in mind when sizing batch/chunk sizes.
Discussion angle
The post's premise is 'do more CPU work to go faster' — worth testing against your own workload: when have you seen a high-cardinality GROUP BY where adding compute made it faster, and how would you tell memory-stall from genuine compute saturation without a profiler?