WALs

WALs, short for write-ahead logs, are a technique used to ensure durability and consistency of data modifications in computer systems, most notably in databases and journaling file systems. The core idea is to record every intended change to a log before applying it to the primary data store, so that after a crash the system can recover by reprocessing the log.

In a typical WAL implementation, each log record includes a log sequence number, a transaction identifier, the type of operation, and enough information to redo or undo the change. Writes are appended to the log in a durable manner, and the corresponding data pages are updated afterward. A commit usually guarantees that the relevant log entries have been persisted to stable storage, which ensures durability of committed transactions.

Recovery uses the WAL to reconstruct a consistent database state after a failure. By replaying the log, the system can redo committed transactions and, if necessary, undo uncommitted ones. Many systems also support point-in-time recovery by applying archived WAL segments up to a chosen moment. WALs are also used in replication, where a primary system streams log records to replicas, which replay them to stay synchronized.

Examples and contexts: PostgreSQL relies on a robust WAL to enable crash recovery, PITR, and streaming replication. SQLite offers a WAL mode as an alternative to its rollback journal. Journaling file systems and other databases employ similar log-based approaches to protect metadata integrity and aid recovery.

Considerations for managing WALs include the I/O overhead of writing logs, the need for log retention and archiving policies, and the storage required for log segments. Proper configuration enables effective backups, recovery, and replication, while mismanagement can impact performance and disaster recovery.