Ratelimiting

Ratelimiting is a technique used to control the rate at which clients can interact with a service. Its primary goals are to protect resources, prevent abuse, and ensure predictable performance and availability for all users.

Limits are typically defined as a maximum number of requests or actions allowed within a time window. Enforcement can occur at edge points such as API gateways and reverse proxies, or within applications themselves. Common algorithms include fixed window, sliding window, token bucket, and leaky bucket. The fixed window method counts requests within a fixed time interval but can create boundary bursts; sliding window methods smooth those bursts; token bucket and leaky bucket allow controlled bursts up to a capacity while maintaining an average rate.

Ratelimiting can be applied at various granularities, including per IP address, per API key, per user, per endpoint, or per combination of these. The most common response to exceeding a limit is an HTTP 429 Too Many Requests status, often accompanied by a Retry-After header indicating when to retry.

Implementation considerations include the challenges of distributed systems, such as synchronizing state across multiple instances or relying on centralized stores like Redis. Local counters or distributed counters each have trade-offs in accuracy and latency. Common pitfalls involve clock skew, unintended exemptions, or inconsistent enforcement across services.

Operational practices emphasize clear error signaling, informative retry guidance, and monitoring of metrics such as request rate, 429 rate, and latency. Best practices also include gradual rollouts, adaptive limits based on load, and combining rate limiting with other resilience strategies like backoff, circuit breaking, and caching.