SpanCore

SpanCore is a modular computing framework designed to simplify the development and deployment of real-time data processing applications. It introduces the concept of a span, a unit of work that encapsulates a small computation along with its input and output data and its dependency constraints. Spans can be scheduled independently or as part of a dataflow graph, enabling scalable execution on multi-core CPUs and accelerators.

Architecture and components: The framework comprises a Span Definition Language (SDL) for describing spans and their dependencies; a SpanCore runtime responsible for scheduling and coordinating execution; a collection of core span libraries providing common functionality; and adapters to connect to external data sources and sinks. The runtime supports locality-aware scheduling to improve cache reuse and energy efficiency and can operate in single-node or distributed configurations. SpanCore emphasizes portability, with reference implementations available for multiple languages and environments.

Programming model: Developers implement spans as lightweight functions or closures that declare input and output data, which the runtime routes through the graph. Spans declare data dependencies and timing constraints; the runtime handles data passing, synchronization, and fault isolation. The model favors small, testable units of work and aims to minimize cross-span data movement.

Development and status: SpanCore originated from a collaborative effort in the early 2020s and has evolved through community-driven iterations. It is used in experimental pilots for streaming analytics, telemetry fusion, and edge processing. Adoption remains limited to research groups and early adopters, with ongoing work to standardize interfaces and improve tooling. See also dataflow programming and task-based parallelism.