CSVNetCDF

CSVNetCDF is a data interchange approach and accompanying workflow for converting comma-separated values (CSV) files into the Network Common Data Form (NetCDF). The goal is to preserve tabular measurements while enabling integration with array-oriented scientific datasets and metadata standards such as CF conventions.

Conceptually, CSVNetCDF defines how to map CSV structure to a NetCDF data model. A CSV file may contain a header row naming columns and a data section with rows representing observations or time steps. In a CSVNetCDF translation, certain columns become coordinate variables (for example time, latitude, longitude) and the remaining columns become data variables with appropriate dimensions. Metadata about units, long names, and descriptive attributes is provided either in adjunct files (for example a YAML or JSON metadata document) or in specially formatted header fields.

Workflow typically involves parsing the CSV, defining or inferring dimensions, creating the NetCDF dataset, defining dimensions and variables, attaching attributes, converting values, handling missing data, and writing the NetCDF file. CF-compliant attribute assignments are commonly used to improve interoperability. Popular tooling includes libraries such as Python's netCDF4 and xarray, sometimes complemented by metadata extractors and utility suites that perform CSV-to-NetCDF conversion.

Applications and limitations: CSVNetCDF is useful for archiving sensor logs, experimental data, or outputs from model runs that originate as CSV. Limitations include potential loss of hierarchical structure, ambiguity in mappings without clear metadata, and challenges in efficiently encoding very large CSV files as NetCDF.

Status and adoption: CSVNetCDF is not a formal international standard but a descriptive term used in some data workflows. It emphasizes interoperability between simple CSV sources and NetCDF-based data analysis environments.