lendataframe

lendataframe is a lightweight Python library that provides a data frame abstraction similar to pandas but with a focus on memory efficiency and reduced dependency overhead. Designed for small to medium‑sized data sets that fit comfortably in RAM, lendataframe offers core functionalities such as columnar storage, broadcasting, and grouping, while deliberately omitting some advanced pandas features that contribute to larger memory footprints.

The library is developed under an MIT license and is hosted on GitHub, where it receives contributions from both individuals and organizations. Its API mirrors a subset of pandas’ syntax: construction via a dictionary of column names to lists or NumPy arrays, indexing with boolean masks, and GroupBy operations for aggregation. Internally, lendataframe stores data in a column‑major format that allows contiguous memory allocation, which improves cache locality and speeds up many columnar operations.

Key features include type‑annotated columns, support for mixed dtype columns through a lightweight variant of NumPy’s dtype system, and a minimal set of optimization flags that can be toggled for faster read‑only queries. Documentation is available via an online ReadTheDocs site, and the library is distributed through PyPI under the package name lendataframe.

The project encourages contributions that improve performance, expand supported aggregation functions, or provide better interoperability with existing scientific libraries. It also maintains a small but active community forum for user questions and feature requests. In environments where pandas’ overhead is prohibitive, lendataframe offers a pragmatic alternative without sacrificing the expressive power of columnar data analysis.