Designing a global rate limiter that stays fair under load
Fairness, quotas, and multi-region consistency for a rate limiter that holds up in interviews and in production.
A rate limiter looks simple until you make it global. The moment you have more than one region, you have to choose between strict correctness and low latency, and that choice shapes the whole design.
The core is a counter per key, per window. A token bucket gives you smooth limits and a burst allowance, which is usually what teams actually want. Store the buckets in a fast shared store so that any node can enforce the same limit.
The hard part is consistency across regions. Strict global counting means a synchronous check on every request, which adds latency. A common tradeoff is to allow each region a share of the budget and reconcile asynchronously. You accept a small overshoot at the edges in exchange for fast local decisions.
Decide the failure mode on purpose. When the shared store is unreachable, do you fail open and let traffic through, or fail closed and reject it. For abuse protection, failing closed is safer. For a paid API where availability matters, failing open with a local fallback limit is often better.
Key points
- ·Token bucket gives smooth limits with a burst allowance.
- ·Split the budget per region and reconcile async to keep latency low.
- ·Choose fail-open versus fail-closed deliberately.
Sources
Replies
Sign in to reply.
No replies yet. Be the first to add something useful.