Zst

Zstandard, also known as zstd, is a fast lossless data compression algorithm and file format designed by Facebook (Meta) and released as open source. The associated file extension is .zst. It is widely used for efficient storage and transmission of data.

The standard uses a framing format that supports streaming and multiple frames. Each frame contains a header, a compressed payload, optional metadata, and a checksum. This design enables robust error detection and allows flexible handling of large or streaming data.

The compression engine blends LZ77-style matching with an efficient entropy coder and adaptive modeling. It offers a broad spectrum of compression levels, typically from 1 to 22, allowing users to trade speed for ratio, and supports dictionaries to improve results on small or repetitive data.

Tooling and APIs: The zstd command-line utility and libzstd C API are the reference implementations. The format is used for archiving, packaging, and data transmission, and supports streaming, multi-threaded compression and decompression, and progressive decoding.

Adoption and usage: Zstandard is widely used in software packaging and distribution, and has been adopted by several Linux distributions, backup tools, and cloud services. It competes with gzip, bzip2, and xz by offering faster speeds with competitive compression ratios, especially on modern hardware.

History and licensing: Zstandard was developed by Yann Collet and released publicly in 2016. It is open source under permissive terms, with ongoing development and broad ecosystem support.