projx

projx is an open-source framework for data processing and machine learning workflow orchestration. It provides modular components for data ingestion, transformation, model training, evaluation, and deployment, with a focus on reproducibility and extensibility.

Origin and governance: projx was initiated in 2016 by a consortium of universities and industry partners. It is developed by a community-driven project with contributors worldwide. The project is licensed under the MIT License.

Architecture: The core engine is modular, featuring a plugin system for data connectors, compute backends, and model registries. Pipelines can be defined with a Python DSL or YAML, and projx supports batch and streaming workloads, on CPU or GPU backends.

Features: It includes data connectors, transformation steps, experiment tracking, lineage, and a model registry. It offers a command-line interface and REST API, and can deploy trained models to serving platforms. It emphasizes reproducibility through experiment metadata and versioned artifacts.

Adoption and reception: projx is used in research labs and industry for building reproducible ML pipelines. It has an active ecosystem of plugins and tutorials, though some users note a learning curve and overlap with other workflow tools.

See also: MLflow, Apache Airflow, Kubeflow.