filepertable

Filepertable is a data storage pattern in which each table of a dataset is stored as a separate file within a filesystem or object store. The pattern is commonly used in lightweight data architectures, data exchange pipelines, and embedded applications where a full database management system is not desirable or feasible. In a typical implementation, a dataset is organized as a directory or bucket containing multiple files, each file representing one logical table. File formats vary and may include CSV, JSON, Parquet, or binary encodings; a metadata catalog records the table names, column definitions, keys, and relationships to enable cross-file queries.

How it works: an external query engine or application that understands the catalog reads the relevant table files and performs operations such as filtering, projection, and joins by accessing multiple files. Integrity constraints such as foreign keys or transactions are not guaranteed by default and must be enforced at the application level or by a coordination layer. Partitioning, data versioning, and schema evolution are potential benefits and challenges depending on the chosen formats and tooling.

Advantages of the filepertable pattern include isolation between tables, easier export and interoperability, and simpler schema changes for individual tables. Disadvantages include higher latency for multi-table joins, the lack of built-in transactional semantics, potential metadata drift, and heavier tooling requirements for consistent querying.

See also: one-file-per-table, data lake, data warehouse, schema registry.