JetArrays

JetArrays is a high-performance software framework and data structure designed for organizing and processing large streaming datasets. It introduces the concept of a jet, a lightweight subarray that represents a contiguous block of data associated with a parent array. Jets enable composable, streaming computations with minimal copies and predictable memory usage.

In JetArrays, data are stored as JetArray objects, consisting of a collection of jets along one or more axes. Each jet carries its own length and stride information, allowing operations to be applied to entire jets or individual elements. The framework supports lazy evaluation, so computations are chained and executed only on demand. Core operations include map, filter, reduce, join, and group-by, designed to preserve jet boundaries for parallel execution.

The execution model is data-flow oriented. JetArrays can run on multi-core CPUs and can target accelerators through backends, enabling vectorized execution. The memory model emphasizes zero-copy slicing and view-based access to minimize allocations during pipelines.

Origin and status: JetArrays began as a research concept to address high-throughput event streams in scientific and financial domains. It has evolved into an open-source project with bindings for several languages and ongoing community development.

Applications: real-time analytics, sensor and event streams, simulations, and time-series processing in scientific computing and finance.

Limitations: best suited for streaming workloads with regular jet shapes; limited benefit for static, random-access workloads.

See also: data structures, streaming data processing, arrays.