loadlimiting

Loadlimiting refers to techniques that keep a system from becoming overloaded by actively restricting incoming work as demand grows toward capacity. The goal is to maintain acceptable latency and error rates by capping, delaying, or prioritizing work before queues become saturated.

Common methods include rate limiting (such as token bucket or leaky bucket), admission control to reject or defer requests, throttling to slow processing, and backpressure that signals upstream components to reduce throughput. Circuit breakers can temporarily stop traffic to failing services. Prioritization and queue management help ensure critical tasks proceed while less important work is delayed or dropped. Implementation may occur at API gateways, services, message queues, or storage layers.

Effective load limiting relies on monitoring state and performance targets. Thresholds should reflect service level objectives and adapt to changing load. Observability, tracing, and alerting reduce the risk of over- or under-reaction. The approach trades off availability, latency, and throughput, and must avoid excessive user disruption.

Applications include web APIs, cloud services, streaming pipelines, and databases, where load limiting preserves responsiveness during peak demand or partial failure. It complements, but is distinct from, load shedding (dropping requests) and auto-scaling.

See also: rate limiting, backpressure, circuit breaker, load shedding.