Mergeonread

MergeonRead, abbreviated MOR, is a data storage and processing approach that defers merge operations to read time, allowing incoming updates to be appended as lightweight delta records rather than performing immediate upserts. The technique is commonly applied in log-structured storage and data lake architectures to balance write throughput with query freshness.

Mechanism: In MOR, writes are stored as discrete delta entries or log records that describe changes relative to a base dataset. When a query or read occurs, a merge step applies the relevant deltas to reconstruct the current view. Depending on the implementation, MOR can be paired with periodic compaction or indexing to optimize repeated reads and reduce latency.

Applications: MOR is used in large-scale data ingestion pipelines, event streams, and analytic workloads where updates are frequent and immediate strong consistency is less critical. It is often discussed in the context of data lake storage engines and formats that support incremental updates, enabling scalable ingestion while maintaining queryable state.

Advantages and limitations: Benefits include higher write throughput, lower write amplification, and simpler append-only ingestion. Drawbacks include higher read latency for complex queries, potential read amplification, and the need for maintenance tasks such as compaction, indexing, or specialized query planning to keep reads efficient.

History: The concept aligns with broader log-structured and append-only design principles in data management. MOR is related to, but distinct from, merge-on-write approaches that update data by rewriting records. In practice, implementations vary across systems and data models.

See also: Merge-on-read, Merge-on-write, data lake storage engines.