kihist

kihist is a software library and data format designed for efficient histogram construction, storage, and merging in streaming and distributed analytics environments. It aims to provide scalable distribution summaries for high-volume data streams, enabling real-time telemetry, monitoring, and analytics workflows.

The data model uses a compact representation of histogram buckets and counts. Histograms can use fixed buckets for known data ranges or adaptive buckets for dynamic ranges. The project supports both exact histograms for small domains and approximate histograms via mergeable sketches for large-scale data, trading exactness for memory and speed.

Ingestion and processing: kihist exposes streaming APIs for high-rate data ingestion, with optional windowing and time-based aggregation. Histograms can be merged across threads, processes, or machines, enabling distributed computation. It provides percentile and quantile estimates with configurable error bounds and supports weighted samples.

Implementation and availability: the core is written in C++ with language bindings for Python, Java, and Go, allowing use in common data pipelines. The library includes a compact persistence format for interchange and storage, as well as tools for exporting to JSON or other interoperable representations and for integrating with metric backends.

History and use cases: kihist originated in open-source observability and data-analytics projects to fill the need for scalable distribution summaries in modern services. It is used for latency tracking, anomaly detection, capacity planning, and sensor data analysis in cloud, edge, and IoT environments.

See also: histograms, mergeable data sketches, streaming analytics, telemetry.