This visual model shows how the CPU searches through progressively larger and slower cache levels before going to main memory.
CPU Core
|
v
┌─────────────────┐
│ L1 │
│ smallest │
│ fastest │
└─────────────────┘
|
miss
|
v
┌─────────────────┐
│ L2 │
│ larger │
│ slightly slower│
└─────────────────┘
|
miss
|
v
┌─────────────────┐
│ L3 │
│ larger still │
│ often shared │
└─────────────────┘
|
miss
|
v
┌─────────────────┐
│ Main Memory │
│ RAM │
│ much slower │
└─────────────────┘
The CPU checks the fastest and smallest cache first. If the needed data is found there, the access is fast. If not, the CPU checks the next cache level.
Each miss moves the request farther away from the CPU and usually makes the access slower. Most fast programs benefit from keeping frequently used data close to the CPU in cache.
The goal of the cache hierarchy is simple: avoid going to RAM whenever possible.