cachemiss

A cache miss is an event in a computer system when the data or instruction requested by the processor is not found in the cache memory. When a miss occurs, the processor must fetch the data from a lower level of the memory hierarchy, such as the next cache level or main memory, introducing a delay known as the miss penalty. The overall performance impact depends on the cache's hit rate and the relative cost of cache accesses versus memory accesses.

Misses are typically categorized into compulsory (cold) misses, capacity misses, and conflict misses. Compulsory misses occur on the first access to a data block that has not yet been loaded into the cache. Capacity misses arise when the cache cannot hold the working set of data, forcing eviction of blocks that will be reused later. Conflict misses occur in set-associative caches when multiple blocks compete for the same cache set, causing evictions even if free space exists elsewhere. In multicore systems, cache-coherence traffic can also trigger additional misses for shared data.

Mitigating cache misses involves improving data locality and cache-friendly design. Techniques include moving data to improve spatial locality, restructuring algorithms (for example, loop tiling), choosing data layouts that favor contiguous memory access, and using prefetching or compiler hints. Hardware improvements such as larger or more levels of cache, and smarter replacement policies, also reduce misses.

In practice, developers measure cache misses with profiling tools and performance models to guide optimization. Key metrics include the cache miss rate (misses per memory accesses) and the miss penalty, which together determine the program’s memory-bound behavior.