Lakehouse

A lakehouse is a data management paradigm that blends the scalable storage of data lakes with the data management and performance guarantees of data warehouses. It aims to provide a single source of truth for structured, semi-structured, and unstructured data used in analytics, machine learning, and reporting.

In a lakehouse, data resides in a centralized object store (for example S3, ADLS, or GCS) and is organized into tables backed by a metadata and transaction layer that provides ACID properties, schema enforcement, and governance. Open formats like Parquet, ORC, and AVRO are common; table formats such as Delta Lake, Apache Iceberg, and Apache Hudi provide transactional semantics.

Key capabilities include ACID transactions, schema evolution and enforcement, time travel or data versioning, and governance features like access control and auditing. The architecture is designed to support both batch processing and streaming workloads with consistent results and reduced data duplication.

Compute is decoupled from storage, enabling multiple engines to query the same data. Popular engines include Spark, Presto/Trino, and various SQL engines, often complemented by data catalogs and query optimizers. This setup supports BI, data science, and ML workflows on a single dataset.

The lakehouse concept emerged in the late 2010s and gained prominence as organizations sought unified analytics platforms. Open-source projects and vendors have contributed implementations, influencing modern cloud data platforms that aim to remove data silos while preserving performance.

Trade-offs include architectural complexity and governance requirements, potential vendor lock-in with proprietary features, and the need for strong data quality and lineage tooling. When well managed, lakehouses can improve data access and accelerate analytics across teams.