h5py

h5py is a Pythonic interface to the HDF5 binary data format. It provides a Python API for reading and writing HDF5 files, mapping the HDF5 structure of files, groups, datasets, and attributes to Python objects. It is widely used in scientific computing to store large, heterogeneous data.

The library integrates closely with NumPy; datasets behave like NumPy arrays with indexing, slicing, and broadcasting. Data can be read or written in whole or in chunks, and h5py supports data compression and chunked storage via HDF5 filters such as GZIP and LZF. Data types map between NumPy dtypes and HDF5 dtypes, enabling straightforward interoperability with NumPy arrays.

The API supports common operations: opening a file with h5py.File, creating groups and datasets with specified shape and dtype; extendable datasets with maxshape; resize; and attributes via the attrs attribute on files, groups, or datasets. Links and other HDF5 features are accessible through the API, providing a familiar file-system-like hierarchy.

Performance and use cases: designed for efficient I/O of large arrays and datasets, enabling partial read/write and hierarchical organization. It is suitable for scientific datasets, imaging data, time series, and simulations. Careful selection of chunking and compression can improve access patterns and storage efficiency.

Usage and ecosystem: maintained as an open-source project; available on Python packaging platforms and documented at h5py.org. It is commonly used alongside NumPy, SciPy, and other scientific libraries. Alternatives include PyTables, which provides a higher-level, table-oriented interface to HDF5 data.