CacheFriendly

Cache-friendly is a term used to describe software design and algorithms that are optimized to take advantage of CPU caches in order to improve performance. The aim is to increase data locality so that the processor spends less time waiting for memory and more time performing computations. Cache-friendly code seeks to maximize cache hits across the cache hierarchy (L1, L2, L3) and to minimize cache misses and memory bandwidth bottlenecks.

The underlying idea rests on locality of reference: temporal locality, the reuse of recently accessed data, and spatial locality, access to data located nearby in memory. By organizing data and computations to exploit these properties, programs can run faster on modern hardware. Techniques commonly used include arranging data in contiguous layouts (for example, arrays instead of linked structures), iterating with small, often unit, strides, and employing tiling or blocking to keep working data sets within a cache level during loops. Other methods include loop interchange to improve access patterns, data alignment to cache lines, and, where appropriate, prefetching hints or compiler optimizations.

Examples and domains where cache-friendly practices matter include numerical linear algebra (such as matrix multiplication with loop tiling), sorting and data processing, and systems or database code that processes large buffers. Cache-friendly design often involves trade-offs with readability, portability, and portability across different hardware, since optimizations can depend on cache size and line length. Cache-oblivious approaches aim to achieve good locality without specific hardware parameters, while cache-friendly techniques typically tailor data layouts and access patterns to known cache characteristics.

See also concepts such as temporal locality, spatial locality, cache lines, and cache-oblivious algorithms.