Rate limiting and quotas

How to cap how fast a client can call you: fixed and sliding windows, token buckets, choosing the key an attacker cannot cheaply multiply, counting across many servers, and what to do when the counter store itself is down.

● EasyFREEBy ·Design problem·12 min read·Beginner → Intermediate
rate limitingtoken bucketsliding window429quotasbrute force

01What rate limiting is for

✓What you will learn
  • Compare fixed window, sliding window and token bucket algorithms and their burst behaviour
  • Choose the limit key (IP, account, API key, tenant, route) so an attacker cannot multiply it cheaply
  • Count across many servers with a shared store or local approximations
  • Decide fail open or fail closed per route when the limiter is unavailable
◎Interview radar

What they probe: which algorithm and why, what the key is for a login endpoint versus a public API, how counters stay consistent across 50 servers, what happens when Redis is down. Where candidates slip: one global per-IP limit for everything, locking accounts after five failures (which lets anyone lock out anyone), no answer for a down counter store.

A rate limit caps how many requests a client may make in a period, for example 100 per minute. A quota is the same idea over a longer period and usually tied to a plan, for example 1 million API calls a month. Both protect three things: availability (one client cannot starve others), security (guessing passwords, codes or IDs becomes slow) and cost (expensive operations like sending SMS stay bounded).

UseExample limitKey
Login10 failures per account per 15 min, plus per-IP and per-fingerprint limitsAccount and source together
One-time code check5 attempts per code, then the code is deadCode or challenge ID
Public API1,000 requests per minuteAPI key or tenant
Password reset email3 per account per hourAccount and IP
Expensive searchCost units per minute, not request countUser

02The algorithms

AlgorithmHow it worksBurst behaviourMemory per key
Fixed windowCounter per key per minute bucket, reset at the boundaryUp to 2x the limit across a boundary (end of one minute plus start of the next)One integer
Sliding window logStore every request timestamp, count those within the last N secondsExactOne entry per request: expensive at high limits
Sliding window counterWeighted sum of the current and previous fixed windowsClose to exact, small errorTwo integers
Token bucketBucket holds up to B tokens, refills at R per second; each request takes oneAllows a burst of B, then the steady rate RToken count plus last refill time
Leaky bucketRequests join a queue drained at a fixed rateSmooths output, adds delayQueue length
Why fixed windows bursttxt
fixed window, limit 100 per minute
  12:00:59  client sends 100  -> allowed (window 12:00)
  12:01:00  client sends 100  -> allowed (window 12:01)
  -> 200 requests in about one second, double the intended rate

sliding window counter at 12:01:15 (25 % into the current window)
  previous window count 100, current 20
  estimate = 100 x (1 - 0.25) + 20 = 95  -> 5 more allowed
Token bucket in a dozen linests
/** Token bucket: capacity B, refill R tokens per second. State is two numbers per key. */
export interface Bucket { tokens: number; updatedMs: number }

export function take(b: Bucket, nowMs: number, capacity: number, refillPerSec: number, cost = 1): boolean {
  if (capacity <= 0 || refillPerSec <= 0 || cost <= 0 || cost > capacity) throw new RangeError("bad limiter config");
  const elapsed = Math.max(0, nowMs - b.updatedMs) / 1000;   // ignore clock going backwards
  b.tokens = Math.min(capacity, b.tokens + elapsed * refillPerSec);
  b.updatedMs = Math.max(b.updatedMs, nowMs);
  if (b.tokens < cost) return false;                          // caller returns 429 with Retry-After
  b.tokens -= cost;
  return true;
}

When a request is refused, return HTTP 429 Too Many Requests (RFC 6585) with a Retry-After header so well-behaved clients back off. For login and code checks, consider returning the same response as a failure so attackers cannot tell when they have been limited.

03Deep dive: choosing the key and counting everywhere

Pick the key the attacker pays for

  • Per IP is cheap to evade: residential proxy pools sell tens of thousands of addresses, and mobile carrier NAT puts many real users behind one IP.
  • Per account stops guessing one user's password, but a hard lockout lets anyone lock out any user. Prefer growing delays and step-up challenges over a lock.
  • Per API key or tenant is fair for paid APIs, because keys are issued, not free.
  • Composite keys (account plus IP, route plus tenant) and several limits at once cover more attack shapes than any single key.
  • Cost-weighted limits charge expensive calls more tokens, for example a GraphQL query by its computed cost rather than as one request.

Counting across many servers

ApproachHowTrade-off
Central storeEach request does an atomic increment (for example a Redis INCR with expiry, or a Lua script for a token bucket)Accurate; adds a network hop (about 1 ms) and the store becomes critical
Local limitsEach of N servers allows limit / NNo hop; inaccurate when load is uneven across servers
Local with periodic syncServers count locally and push deltas every secondSmall overshoot during the sync interval; scales well
Edge plus centralCoarse per-IP limit at the edge, precise per-account limit centrallyCheap traffic filtered early, precision where it matters
Sizing a central limitertxt
API traffic             50,000 requests / s
limit checks            1 per request, 2 Redis ops each (increment + expiry)
  -> 100,000 ops / s
one Redis shard         on the order of 100,000 simple ops / s
plan                    shard by key hash across 4 shards for headroom
memory                  2 M active keys x 100 B = 200 MB  (trivial)

04Failure modes and trade-offs

SituationChoiceWhy
Counter store down, public read APIFail open with a local per-server limitAvailability matters more than precision
Counter store down, login or code checkFall back to a strict local limit, never unlimitedAn outage must not become a free brute-force window
Clock skew between serversUse the store's time or token buckets that tolerate small skewWindows misalign otherwise
Hot key (one tenant at huge volume)Local pre-aggregation for that keyOne shard melts otherwise
!Lockout is a denial of service you built

"Lock the account after five wrong passwords" lets an attacker lock out your CEO, or every user, with a username list. Combine soft per-account delays with per-source limits and step-up, and let the real user in through a verified channel.

iRate limits are not authorisation

A limit slows down guessing; it does not make a guessable thing safe. Sequential IDs, six-digit codes without expiry and short tokens need fixing at the source, with the limit as a second layer.

05Summary, follow-ups and self-test

▭On the whiteboard · Remember3 points
01
algorithm
token bucket for APIs (burst + rate); sliding window counter when bursts are unwanted
02
key
the identity the attacker pays for; several limits at once
03
outage
fail open for reads, strict local fallback for login and codes
“How to say it in the room

"For the public API I'd use a token bucket per API key, so clients can burst a little but average out at their plan rate, checked against a sharded Redis with an atomic script. Login gets several limits at once: per account with growing delays rather than a lockout, plus per IP and per device fingerprint, because proxy pools make IPs cheap. If Redis is down, reads fail open to a local limit, but login falls back to a strict local limit so an outage is never a free brute-force window."

Q1How would you rate limit a GraphQL endpoint?+

Compute a cost for each query from its depth and the number of objects it can return, cap depth and cost per query, and debit cost units from the caller's bucket instead of counting requests.

Q2A big customer says your limit is hurting their batch job. Options?+

Raise their tenant quota, give them a bulk endpoint, or let them buy a higher tier. Do not exempt them by IP.

Q3Why return the same response when a login is rate limited?+

A distinct response tells the attacker when to rotate proxy or slow down, and confirms the account exists. Keep the response and timing identical to a failed login where possible.

Test yourself

A fixed-window limiter allows 100 requests per minute per key. What is the most a client can send in a two-second span?

Finished this one?
Tick it and it shows as done on the roadmap and the course.