Cache

A cache is a fast, smaller storage that holds a subset of data from a larger, slower data store to speed up future requests. It leverages temporal locality (recently accessed data) and spatial locality (data near recent items) to reduce access latency.

Caches exist at many levels. CPU caches (L1, L2, and sometimes L3) are small but very fast memories near the processor that store frequently used instructions and data. Operating systems maintain page caches for disk data in memory. Web caches in browsers, proxies, and content delivery networks serve commonly requested resources nearer to users.

Performance is described by hit rate and miss rate. A cache hit returns data quickly; a miss requires retrieving data from a slower backing store. Warm caches perform better than cold caches, and cache effectiveness depends on workload and size.

Cache management uses replacement and write policies. Replacement algorithms decide which item to evict when the cache is full (LRU, LFU, FIFO are common). Write policies determine how updates propagate to the back-end store, such as write-through or write-back. Prefetching can improve utilization by loading data ahead of requests.

Coherence is a concern in multi‑processor systems. Cache coherence protocols ensure that all processors observe a consistent view of memory, using techniques such as invalidation or update messages (for example, the MESI protocol).

Caches are a source of speed but add complexity and potential data staleness. Designing a cache involves trade-offs among size, speed, consistency, and overhead.