fetcher

A fetcher is a software component or function that retrieves data from a source for use by an application. Fetchers encapsulate the details of data access, enabling other parts of the system to request information without needing to know the origin, transport protocol, or query mechanics.

In web development, a fetcher typically issues a request to a server or API and returns the result. Common implementations include the browser fetch API, as well as server-side libraries such as curl, requests, httpx, axios, and similar HTTP clients. In front-end frameworks, a fetcher function often powers data loading for components, sometimes with built-in support for caching, revalidation, and error handling.

In data engineering and ETL, a fetcher pulls data from diverse sources such as relational or NoSQL databases, object storage, file systems, or streaming services. It may support batch or streaming modes, handle pagination or cursor-based reads, and normalize data formats for downstream processing.

Key design considerations include idempotence, reliability, and performance. Fetchers should define clear timeouts, retry policies with backoff, and error handling strategies. Caching and stale-data policies influence freshness guarantees, while authentication, encryption, and access control protect data in transit and at rest. Concurrency and rate limiting affect throughput and fairness when multiple fetchers access the same resource.

Overall, fetchers are a fundamental building block for modular, data-driven software, enabling flexible data retrieval across diverse sources.