Columnbased

Columnbased, often described as columnar or column-oriented storage, refers to data organization where data is stored and processed by columns rather than by rows. In a columnbased system, each column of a given dataset is stored contiguously on disk or in memory, and queries access only the relevant columns needed for a computation. This approach is used in analytical databases, data warehouses, and modern file formats.

How it works: Data types are stored column-wise; values from a single column are stored adjacent to each other, enabling high compression and fast scans. Columnar layouts support encoding schemes optimized per data type and predicate pushdown, allowing engines to skip entire blocks when a predicate is false. In-memory columnar representations also enable vectorized execution, processing many values in parallel.

Advantages and use cases: Column-based designs excel at read-heavy analytic workloads and aggregation queries over large datasets. They typically achieve higher compression ratios and reduced I/O, leading to faster scan times and throughput. They are common in data warehousing, business intelligence, and machine learning pipelines that read wide tables but touch only a subset of columns.

Trade-offs and limitations: Columnar storage can compromise write performance, random updates, and row-wise transactional workloads. It is less efficient for small, random-access updates and online transaction processing. Practical deployments often combine column-based storage with row-based components or use hybrid systems. Notable examples include columnar databases such as ClickHouse and Amazon Redshift, and columnar file formats like Apache Parquet and ORC, which are widely used by data processing engines.