PipelineResources

PipelineResources refers to the set of external assets that a pipeline interacts with to produce results. In pipeline-based workflows, these resources act as inputs, outputs, or interfaces to external systems, ranging from code repositories and data sources to artifact storage and service endpoints. PipelineResources are defined separately from the tasks or steps that operate on them, enabling modularity and reuse across different pipelines.

A typical PipelineResources definition specifies a name, a resource type, and configuration parameters that tell the pipeline how to communicate with the external system. The resource type determines the protocol and operations supported, such as fetching latest data, storing artifacts, or updating external services. Common operations associated with resources include check (to detect new versions or updates), get (to retrieve input data or artifacts), and put (to publish output data or artifacts). Some implementations also support peek, which reads metadata without altering the resource state.

Usage and lifecycle emphasize decoupling: pipelines declare their resources, while runtime systems manage authentication, versioning, and state. This separation improves reproducibility, traceability, and parallelism, as tasks can operate on stable inputs or publish outputs to well-defined destinations.

Examples of PipelineResources include git repositories as sources of code, object storage buckets for artifacts, container registries for images, and databases or message queues as data inputs or sinks. Security considerations, such as access control and credential management, are important, given that resources often interface with external systems.

In summary, PipelineResources provide the loosely coupled interfaces that connect a pipeline to its external world, enabling controlled, repeatable, and observable automation.