dataf

Dataf refers to a data interchange format and accompanying software libraries that have appeared in computer science literature and open-source experiments as a concept for efficient data ingestion, storage, and interchange in data-intensive applications. It is not an official standard and has limited adoption outside experimental and educational contexts.

Design goals of dataf include fast serialization and deserialization, compact binary encoding, and robust support for evolving schemas. The format is typically described as pluggable between binary and compressed representations, with an emphasis on low-latency processing in streaming pipelines. Dataf supports common primitive types, nested records, arrays, and maps, and it provides metadata facilities to describe schemas, provenance, and related constraints. Language bindings exist for several programming languages, enabling integration with popular data-processing stacks.

In practice, dataf is positioned as a lightweight alternative to heavier columnar formats in scenarios where streaming latency is critical, and as a bridge between row-oriented and column-oriented systems. While it shares ideas with established formats such as Parquet, Avro, and Protocol Buffers, dataf aims for closer alignment with real-time analytics and message-streaming platforms.

Adoption remains limited, and toolchains are mostly research-oriented or educational. For projects requiring formal standardization or broad ecosystem support, more mature formats are typically preferred. See also Parquet, Avro, Protocol Buffers, JSON, and ORC.