Why is CPU cache faster than RAM? Proven in one number you can reproduce

Why is CPU cache faster than RAM? One reproducible number#

Most explainers stop at "cache is closer and smaller". That is true, yet it explains nothing you can act on. Instead, start with the measurement and work backwards to its cause. On a Skylake i7-6700 an L1 hit costs about 1 nanosecond, and a DRAM read costs about 61.5. That single ratio is the whole thesis, and every section below earns it.

The claim, and how the rest of this page proves it#

So why is CPU cache faster than RAM? It comes down to two independent penalties that add up. The first is the cell. The second is the distance. Because both are physical, both are measurable, and neither is a vendor talking point. The chain runs in five links. First the transistor, then the DRAM timing, next the on-die signal path, then a benchmark you run, and finally a table that maps each symptom to a fix. Read the map once and no later section is a surprise.

The latency ladder: cycles, nanoseconds, capacity, and bandwidth#

Before the physics, here is the ground truth every top result leaves scattered across ten pages. This table gathers the whole hierarchy for one named chip. Each row lists the latency in cycles and nanoseconds, the typical capacity, and a rough bandwidth. Read it top to bottom and the cost of distance is already visible.

Reading each row: the physical cause behind the number#

That single table is the spine of why CPU cache is faster than RAM. Notice the shape of the jump. From L1 to L3 the latency grows about tenfold, yet it stays on-die. From L3 to DRAM it grows sixfold again, because the request now leaves the chip. Meanwhile the capacity moves the opposite way, from kilobytes to gigabytes. That trade is not an accident. It is forced by the storage cell, which is where the next section begins.

It starts at the transistor: SRAM's 6T latch vs DRAM's 1T1C cell#

The deepest reason why CPU cache is faster than RAM lives in the storage cell itself. Cache and RAM store a bit with different circuits, and that single choice sets everything downstream. So look at the two cells side by side. One is built for speed, the other for density, and you cannot have both in one cell.

The 6T SRAM cell: a latch you read directly#

An SRAM cell is a latch. Two inverters feed each other, so the pair holds one stable bit as long as power is on. Because the bit is held actively, nothing has to be refreshed. Furthermore the value is read straight off the latch through two access transistors, with no reconstruction step. That is why a cache hit resolves in a handful of cycles. The cost is size. Six transistors per bit is bulky and power-hungry, so SRAM stays measured in kilobytes.

The 1T1C DRAM cell: one capacitor, and why it leaks#

A DRAM cell stores a bit as charge on one capacitor, gated by one transistor. One transistor plus one capacitor is tiny, so DRAM packs gigabytes onto a die cheaply. However that density has a price. Charge leaks away within milliseconds, so the controller must refresh every row on a fixed timer or the data rots. Moreover reading the cell drains the capacitor, which makes the read destructive. Consequently every read is followed by a write-back, and that hidden work is where the stall begins.

Where the stall is born: RAS, CAS, precharge, refresh#

Competitors say "DRAM is slower" and stop. Instead, watch the slowness happen. A DRAM read on a fresh row is not one action. It is a sequence: activate the row with a row address strobe (RAS), select the column with a column address strobe (CAS), move the bytes, precharge the row back, and share time with refresh. Step through it below and watch the roughly 51 nanosecond penalty accumulate, one phase at a time.

None of that work exists for a cache hit. An SRAM latch has no row to activate, no column to select, and no charge to restore. Therefore the DRAM penalty is not a bigger version of a cache access. It is extra machinery that a cache simply does not run.

Distance is latency: the on-die signal path#

Distance is the second half of why CPU cache is faster than RAM. Even at the speed of light, signals take time to cross a chip, and more time to leave it. So the physical layout of the parts sets a floor on latency that no cleverness removes. Trace the path from the execution units out to DRAM and the cost of each hop becomes concrete.

Why L1 sits beside the ALU and DRAM sits across a bus#

L1 is tiny for a reason. A small array can sit right next to the execution units, so its wires are short and its lookup is quick. As the caches grow they must sit farther out, which is why L2 and L3 each cost more cycles. Past L3 the picture changes in kind, not degree. Now the request queues at the memory controller, waits its turn for the shared bus, and travels off-chip. Because that arbitration and travel are serial, they add cycles that on-die caches never pay.

Prove it on your machine: the latency-ladder benchmark#

Numbers you cannot reproduce are just claims. So here is the load-bearing part. A short pointer-chase benchmark measures access latency at every working-set size, and its output draws the same L1, L2, L3, and DRAM knees the table predicts. Build it, run it, and compare your knees to the plot.

The pointer-chase microbenchmark, line by line#

The trick is a dependent load. Each step reads a pointer that tells you the next address, so the CPU cannot prefetch ahead. As the working set grows past each cache, the average access falls out of that level and latency jumps. The strided variant does the opposite job. It exposes conflict misses by hammering the same cache set with a power-of-two stride. The third tab reads the cache topology straight from an Apple M1.

// latency.c -- a pointer-chase microbenchmark.
// Build: cc -O2 -o latency latency.c
// Run:   ./latency
#include <stdio.h>
#include <stdlib.h>
#include <time.h>

// One dependent load per step: p = *p. The chain is a single random cycle,
// so the prefetcher cannot guess the next address. We measure pure access
// latency, never bandwidth.
static double chase(size_t bytes, long iters) {
    size_t n = bytes / sizeof(void *);
    void **a = malloc(n * sizeof(*a));
    for (size_t i = 0; i < n; i++) a[i] = (void *)&a[i];

    // Sattolo's algorithm links every slot into one big cycle.
    for (size_t i = n - 1; i > 0; i--) {
        size_t j = (size_t)rand() % i;
        void *t = a[i]; a[i] = a[j]; a[j] = t;
    }

    void **p = &a[0];
    struct timespec t0, t1;
    clock_gettime(CLOCK_MONOTONIC, &t0);
    for (long i = 0; i < iters; i++) p = (void **)*p;
    clock_gettime(CLOCK_MONOTONIC, &t1);

    __asm__ volatile("" :: "r"(p)); // keep p live; defeat dead-code removal
    free(a);
    double ns = (t1.tv_sec - t0.tv_sec) * 1e9 + (t1.tv_nsec - t0.tv_nsec);
    return ns / iters;
}

int main(void) {
    for (size_t kb = 4; kb <= 512L * 1024; kb *= 2)
        printf("%8zu KB  %6.2f ns/access\n",
               kb, chase(kb * 1024, 50L * 1000 * 1000));
    return 0;
}

// strided.c -- a strided sweep to expose conflict misses.
// Touch one byte every STRIDE bytes, then wrap. A power-of-two stride maps
// many accesses onto the same cache set, so conflict misses show up as a
// latency cliff even when the whole array fits L2.
static double sweep(size_t bytes, size_t stride, long iters) {
    volatile char *a = calloc(bytes, 1);
    size_t mask = bytes - 1;      // bytes is a power of two
    size_t off = 0;
    long acc = 0;

    struct timespec t0, t1;
    clock_gettime(CLOCK_MONOTONIC, &t0);
    for (long i = 0; i < iters; i++) {
        acc += a[off];
        off = (off + stride) & mask;
    }
    clock_gettime(CLOCK_MONOTONIC, &t1);

    __asm__ volatile("" :: "r"(acc));
    free((void *)a);
    double ns = (t1.tv_sec - t0.tv_sec) * 1e9 + (t1.tv_nsec - t0.tv_nsec);
    return ns / iters;
}

# Apple M1: read the cache topology the OS exposes, then cross-check 7-cpu.
$ sysctl hw.cachelinesize \
         hw.perflevel0.l1dcachesize \
         hw.perflevel0.l2cachesize
hw.cachelinesize: 128
hw.perflevel0.l1dcachesize: 131072     # 128 KB L1 data cache
hw.perflevel0.l2cachesize: 12582912    # 12 MB L2, shared by 4 performance cores

On the Skylake box the pointer chase prints about 1 ns while the working set fits L1, then rises through roughly 3 ns and 10.5 ns, and finally settles near 61.5 ns once the data spills to DRAM. In practice those four plateaus are the knees. They are the physical hierarchy, printed by your own terminal.

The knees you should see (and how to read them)#

The plot below is the interactive twin of that benchmark. Sweep the working-set slider and a marker walks the curve, naming the level and the latency at each size. Switch the CPU preset and the knees jump to that chip's cache sizes, because an Apple M1 has a 128 KB L1 and a 12 MB L2 rather than Skylake's 32 KB and 8 MB. This is where prediction meets measurement.

Same data, 165x slower: locality is the whole game#

Here is the payoff the physics was building toward. Take one 256 MiB array of 32-bit integers, which is far larger than the 8 MB L3. Sum it in order and the run is bandwidth-bound at about 24.9 ms. Then chase the same elements in a random dependent order and the run takes about 4.13 seconds. That is the same arithmetic, roughly 165 times slower, purely from destroying locality.

Cache lines and prefetch: why sequential is nearly free#

Memory never moves one byte. It moves a whole 64-byte line. So when you read one integer, the next fifteen arrive with it, already in L1. Furthermore the hardware prefetcher spots a sequential pattern and fetches the next line before you ask. Because of those two effects, a sequential scan hits L1 about fifteen times out of sixteen, which is the 0.37 ns per element above. Random access defeats both, and you pay full DRAM latency on nearly every step.

Associativity and the power-of-two-stride conflict-miss cliff#

Associativity sounds abstract until a loop hits it. A cache maps each address to a small set of slots. When you walk memory with a power-of-two stride, many addresses fall into the same set, so they evict each other even though the array fits in the cache. That is a conflict miss, and it shows up as a latency cliff in the strided benchmark. The fix is small: offset the array or pad the stride so accesses spread across sets.

AoS vs SoA and false sharing, as measured slowdowns#

Data layout is a latency decision in disguise. Store an array of structs and a loop that reads one field still drags the whole struct through cache. Switch to a struct of arrays and that field becomes contiguous, so a single line now carries eight useful values. Compare the two below, then look at the magnitudes.

// Array of structs: each particle is 64 bytes, but the hot loop reads only x.
// Every step drags 56 bytes of cold fields into cache alongside the one it uses.
struct particle { double x, y, z, vx, vy, vz, mass, pad; };
struct particle *p = malloc(N * sizeof *p);

double sum = 0;
for (size_t i = 0; i < N; i++)
    sum += p[i].x;        // 1 useful field per 64 B cache line

// Struct of arrays: x lives in its own tight array. Now a 64 B line carries
// eight consecutive x values, so the same loop touches 8x fewer cache lines.
struct particles { double *x, *y, *z, *vx, *vy, *vz, *mass; };
struct particles p = alloc_particles(N);

double sum = 0;
for (size_t i = 0; i < N; i++)
    sum += p.x[i];        // 8 useful doubles per 64 B cache line

False sharing is the multi-thread version of the same problem. Two threads that update separate counters on the same 64-byte line force the cache-coherence protocol to bounce that line between cores. As a result, adding threads makes the code slower, not faster. Pad each counter onto its own line and the contention disappears.

From symptom to fix: making the physics actionable#

Physics is only useful when it changes your code. So here is the translation layer no competitor provides. Match the symptom you actually observe to the level that binds you, then apply the fix. Each row is a pattern you have probably already hit.

When NOT to reach for cache optimization#

Locality tuning is not free, and it is not always the answer. It makes code more complex, so spend it only where a measurement says the memory system is the bottleneck. Reach for it in the wrong place and you trade readable code for no speedup.

Further reading and sources#

Every figure here traces to a primary source. Ulrich Drepper's What Every Programmer Should Know About Memory is the canonical treatment of the SRAM and DRAM cells and the refresh timing. The measured latencies come from 7-cpu's Skylake and Apple M1 pages, which I cross-checked against a local sysctl read on this M1. For cache organization and prefetch behavior, Agner Fog's microarchitecture and instruction tables are the reference, alongside the Intel optimization manuals. For a feel of the orders of magnitude over time, Colin Scott keeps the Latency Numbers Every Programmer Should Know interactive.

So, why is CPU cache faster than RAM? Now you can answer it from your own terminal, not from a slogan. This sits next to the rest of the low-level work we write about. If dependent-load timing interested you, our walkthrough of how computers generate random numbers one decision at a time traces a similar hardware-to-API path. The same latency budgeting drives the network, which is why what happens in the milliseconds after you hit a URL is the companion read. And once the working set is a database, the same cache math explains many common MySQL performance bottlenecks and how to fix them.