Smallflow

Smallflow is a lightweight dataflow programming concept and accompanying runtime designed for composing small processing steps into a directed acyclic graph. In a smallflow program, each node performs a discrete operation, and edges carry data between nodes.

Origin and scope: The term is used in computer science discussions and hobbyist ecosystems to describe micro-scale data pipelines, often in embedded systems, edge computing, or rapid prototyping. It contrasts with larger-scale dataflow systems by prioritizing simplicity and low resource usage.

Architecture: A smallflow graph consists of operators, inputs and outputs, and a scheduler that executes nodes when inputs are available. It emphasizes determinism, stateless or controlled-state operators, and lightweight serialization of data. Some implementations support streaming semantics and backpressure.

Applications and ecosystem: Smallflow is used for ETL of small datasets, automation pipelines, sensor data processing, or education to illustrate dataflow concepts. Tooling ranges from minimal libraries in scripting languages to minimal runtimes embedded in device firmware. Common features include dependency tracking, error propagation, and easy testing of individual operators.

Relation to other concepts: Smallflow is related to dataflow programming, DAG-based workflow engines, and reactive programming. It emphasizes modular, reusable components and straightforward reasoning about data movement, but may trade scalability and ecosystem richness for simplicity.