APIlimitering

APIlimitering is the practice of restricting the rate at which clients can make API calls to a backend service. The goal is to protect resources, prevent abuse, ensure predictable performance, and maintain service availability under load.

Core concepts include the rate limit (the maximum number of requests allowed in a given time window), quotas (longer-term allowances such as daily or monthly), and burst capacity. Implementations commonly rely on token bucket, leaky bucket, or various window-based counters (fixed or sliding) to track and enforce usage.

Limiter can be enforced at different layers, such as API gateways, reverse proxies, service meshes, or within application code. In distributed systems, the challenge is to enforce limits consistently across multiple service instances and geographic regions, while accounting for clock skew and state synchronization.

When limits are exceeded, servers typically respond with HTTP 429 Too Many Requests, sometimes accompanied by a Retry-After header. Clients should implement backoff strategies, often with exponential backoff and jitter, to reattempt requests safely and avoid synchronized retries.

Key design considerations include choosing the granularity (per API key, per user, per IP, or per endpoint), balancing fairness and efficiency, observability, and handling state recovery in distributed stores. Managers must also address scalability of counters, potential leakage or abuse of credentials, and appropriate handling of internal versus external traffic.

Common patterns include per-key limits, per-tenant quotas, burst allowances, and adaptive throttling based on observed traffic. The choice of approach depends on factors such as traffic shape, service level objectives, and operational complexity.