DataFrame

A DataFrame is a two-dimensional, labeled data structure that stores data in a tabular form. It consists of rows, identified by an index, and columns, identified by labels. Each column can hold data of a different type, such as numbers, strings, or timestamps, while the overall structure maintains a uniform length across columns.

The concept originated in statistics and data analysis with R's data.frame and has been widely adopted in modern data libraries. Prominent implementations include the pandas DataFrame in Python, the data.frame in R, Spark DataFrame in Apache Spark, Dask DataFrame for parallel computation, and DataFrame types in Julia and other ecosystems.

Key features include labeled axes, alignment on join and merge operations, support for missing values, and efficient columnar storage in many implementations. DataFrames are designed for heterogeneously typed columns and for operations that manipulate rows or columns, such as selection, filtering, aggregation, sorting, stacking, and reshaping.

Common workflows involve creating a DataFrame from dictionaries, lists, or arrays; reading data from CSV, JSON, or SQL sources; and performing analyses with group-by aggregations, pivot tables, merges, and joins. Index and column labels enable intuitive access by name, and slices or boolean masks enable subsetting.

Performance considerations include memory usage and the cost of copying data. Many frameworks offer optimizations such as chunked processing, in-memory compression, and columnar storage formats. Some libraries also support interoperability with arrow memory for efficient transfers.