Rate limiting and quotas
How to cap how fast a client can call you: fixed and sliding windows, token buckets, choosing the key an attacker cannot cheaply multiply, counting across many servers, and what to do when the counter store itself is down.
01What rate limiting is for
- Compare fixed window, sliding window and token bucket algorithms and their burst behaviour
- Choose the limit key (IP, account, API key, tenant, route) so an attacker cannot multiply it cheaply
- Count across many servers with a shared store or local approximations
- Decide fail open or fail closed per route when the limiter is unavailable
What they probe: which algorithm and why, what the key is for a login endpoint versus a public API, how counters stay consistent across 50 servers, what happens when Redis is down. Where candidates slip: one global per-IP limit for everything, locking accounts after five failures (which lets anyone lock out anyone), no answer for a down counter store.
A rate limit caps how many requests a client may make in a period, for example 100 per minute. A quota is the same idea over a longer period and usually tied to a plan, for example 1 million API calls a month. Both protect three things: availability (one client cannot starve others), security (guessing passwords, codes or IDs becomes slow) and cost (expensive operations like sending SMS stay bounded).
| Use | Example limit | Key |
|---|---|---|
| Login | 10 failures per account per 15 min, plus per-IP and per-fingerprint limits | Account and source together |
| One-time code check | 5 attempts per code, then the code is dead | Code or challenge ID |
| Public API | 1,000 requests per minute | API key or tenant |
| Password reset email | 3 per account per hour | Account and IP |
| Expensive search | Cost units per minute, not request count | User |
02The algorithms
| Algorithm | How it works | Burst behaviour | Memory per key |
|---|---|---|---|
| Fixed window | Counter per key per minute bucket, reset at the boundary | Up to 2x the limit across a boundary (end of one minute plus start of the next) | One integer |
| Sliding window log | Store every request timestamp, count those within the last N seconds | Exact | One entry per request: expensive at high limits |
| Sliding window counter | Weighted sum of the current and previous fixed windows | Close to exact, small error | Two integers |
| Token bucket | Bucket holds up to B tokens, refills at R per second; each request takes one | Allows a burst of B, then the steady rate R | Token count plus last refill time |
| Leaky bucket | Requests join a queue drained at a fixed rate | Smooths output, adds delay | Queue length |
fixed window, limit 100 per minute
12:00:59 client sends 100 -> allowed (window 12:00)
12:01:00 client sends 100 -> allowed (window 12:01)
-> 200 requests in about one second, double the intended rate
sliding window counter at 12:01:15 (25 % into the current window)
previous window count 100, current 20
estimate = 100 x (1 - 0.25) + 20 = 95 -> 5 more allowed/** Token bucket: capacity B, refill R tokens per second. State is two numbers per key. */
export interface Bucket { tokens: number; updatedMs: number }
export function take(b: Bucket, nowMs: number, capacity: number, refillPerSec: number, cost = 1): boolean {
if (capacity <= 0 || refillPerSec <= 0 || cost <= 0 || cost > capacity) throw new RangeError("bad limiter config");
const elapsed = Math.max(0, nowMs - b.updatedMs) / 1000; // ignore clock going backwards
b.tokens = Math.min(capacity, b.tokens + elapsed * refillPerSec);
b.updatedMs = Math.max(b.updatedMs, nowMs);
if (b.tokens < cost) return false; // caller returns 429 with Retry-After
b.tokens -= cost;
return true;
}When a request is refused, return HTTP 429 Too Many Requests (RFC 6585) with a Retry-After header so well-behaved clients back off. For login and code checks, consider returning the same response as a failure so attackers cannot tell when they have been limited.
03Deep dive: choosing the key and counting everywhere
Pick the key the attacker pays for
- Per IP is cheap to evade: residential proxy pools sell tens of thousands of addresses, and mobile carrier NAT puts many real users behind one IP.
- Per account stops guessing one user's password, but a hard lockout lets anyone lock out any user. Prefer growing delays and step-up challenges over a lock.
- Per API key or tenant is fair for paid APIs, because keys are issued, not free.
- Composite keys (account plus IP, route plus tenant) and several limits at once cover more attack shapes than any single key.
- Cost-weighted limits charge expensive calls more tokens, for example a GraphQL query by its computed cost rather than as one request.
Counting across many servers
| Approach | How | Trade-off |
|---|---|---|
| Central store | Each request does an atomic increment (for example a Redis INCR with expiry, or a Lua script for a token bucket) | Accurate; adds a network hop (about 1 ms) and the store becomes critical |
| Local limits | Each of N servers allows limit / N | No hop; inaccurate when load is uneven across servers |
| Local with periodic sync | Servers count locally and push deltas every second | Small overshoot during the sync interval; scales well |
| Edge plus central | Coarse per-IP limit at the edge, precise per-account limit centrally | Cheap traffic filtered early, precision where it matters |
API traffic 50,000 requests / s
limit checks 1 per request, 2 Redis ops each (increment + expiry)
-> 100,000 ops / s
one Redis shard on the order of 100,000 simple ops / s
plan shard by key hash across 4 shards for headroom
memory 2 M active keys x 100 B = 200 MB (trivial)04Failure modes and trade-offs
| Situation | Choice | Why |
|---|---|---|
| Counter store down, public read API | Fail open with a local per-server limit | Availability matters more than precision |
| Counter store down, login or code check | Fall back to a strict local limit, never unlimited | An outage must not become a free brute-force window |
| Clock skew between servers | Use the store's time or token buckets that tolerate small skew | Windows misalign otherwise |
| Hot key (one tenant at huge volume) | Local pre-aggregation for that key | One shard melts otherwise |
"Lock the account after five wrong passwords" lets an attacker lock out your CEO, or every user, with a username list. Combine soft per-account delays with per-source limits and step-up, and let the real user in through a verified channel.
A limit slows down guessing; it does not make a guessable thing safe. Sequential IDs, six-digit codes without expiry and short tokens need fixing at the source, with the limit as a second layer.
05Summary, follow-ups and self-test
"For the public API I'd use a token bucket per API key, so clients can burst a little but average out at their plan rate, checked against a sharded Redis with an atomic script. Login gets several limits at once: per account with growing delays rather than a lockout, plus per IP and per device fingerprint, because proxy pools make IPs cheap. If Redis is down, reads fail open to a local limit, but login falls back to a strict local limit so an outage is never a free brute-force window."
Q1How would you rate limit a GraphQL endpoint?+
Compute a cost for each query from its depth and the number of objects it can return, cap depth and cost per query, and debit cost units from the caller's bucket instead of counting requests.
Q2A big customer says your limit is hurting their batch job. Options?+
Raise their tenant quota, give them a bulk endpoint, or let them buy a higher tier. Do not exempt them by IP.
Q3Why return the same response when a login is rate limited?+
A distinct response tells the attacker when to rotate proxy or slow down, and confirms the account exists. Keep the response and timing identical to a failed login where possible.
A fixed-window limiter allows 100 requests per minute per key. What is the most a client can send in a two-second span?
Related lessons
- Design a rate limiter componentLLD · Problems: Advanced componentsPro
- Design a token-bucket rate limiterLLD · Problems: first security componentsFree
- Design an API authorisation gatewayHLD · Design problems: Access & identityFree
- Design a DDoS protection edgeHLD · Design problems: Infrastructure & dataPro