memtables

A memtable is an in-memory data structure used to store recently written data before it is persisted to durable storage in log-structured storage systems. It is a core component of many write-intensive databases and file systems that follow a log-structured approach, such as LSM-tree based databases.

Typically, a memtable holds key-value pairs in a sorted form, using a data structure like a skip list or a balanced tree. This organization supports efficient reads and range queries while writes are being buffered. Writes are first recorded in a write-ahead log or commit log for durability, then inserted into the memtable. When the memtable reaches a configured size or time threshold, it is flushed to disk as an immutable, sorted on-disk structure (often an SSTable or equivalent). After the flush, the memtable is cleared to reclaim memory and accept new writes.

Durability and crash recovery rely on the write-ahead log: changes are persisted there before being visible in the memtable, allowing recovery by replaying the log after a crash if needed. The disk-resident structures created from memtable flushes are later managed by background processes such as compaction, which merge and organize data for efficient reads.

Memtables are typically per data structure unit in a database, such as per table or per column family in systems like Cassandra, and per database in LevelDB/RocksDB. They balance fast write throughput with read latency, since queries may need to consult both in-memory memtables and on-disk tables.

Key considerations include memory usage, flush frequency, and the impact of compaction on performance. Proper sizing and tuning help maintain write throughput while keeping read performance acceptable.