bitpacked

Bitpacked refers to data or representations formed by packing values into the minimal number of bits required, rather than aligning data to whole bytes or larger units. Bitpacking is a technique used to reduce storage space and improve bandwidth when transmitting or storing large datasets.

In practice, bitpacked data often consists of fields with fixed bit widths, such as booleans packed into individual bits or integers stored in 3, 5, or 12 bits. Implementations rely on bitwise operations—shifts, masks, and sometimes dedicated bit-stream readers and writers—to place or extract values from a compact bitstream. Bitpacking can be static, with a fixed layout, or dynamic, using variable-length codes or run-length encoding combined with packing.

Common use cases include memory-efficient in-memory data structures, columnar storage formats and compression schemes, and compact network protocols or serialization formats where bandwidth is at a premium. Bitpacked booleans are a classic example, as are small fixed-width integers used in graphics, multimedia codecs, or database engines for bit-packed indexes and statistics.

Trade-offs involve favorable space savings and potential bandwidth improvements versus increased implementation complexity and CPU overhead. Packing and unpacking add processing cost and can complicate random access, alignment, or portability due to endianness and bit-order conventions. Debugging bit-packed data is often more challenging than working with byte-aligned representations.

Related concepts include bit fields in programming languages, bit streams, and various forms of run-length encoding that incorporate bit packing, as well as tools and libraries that support bit-level I/O and data compression.