Rate Limiting

Intermediate · 16 min read · High-Level Design, Reliability, Security & Operations

Token bucket, leaky bucket, fixed and sliding windows, and distributed rate limiting.

A rate limiter controls how many requests a client can make in a given time window. Requests beyond the limit are rejected (usually with HTTP 429 Too Many Requests), delayed, or queued.

Rate limiting is both a standalone interview question ("Design a rate limiter") and a component you should mention in almost every public-facing design.

Why rate limit

What to limit by

The key you count against determines who gets limited:

The algorithms

A bucket holds up to capacity tokens and is refilled at a fixed rate (e.g., 10 tokens/second). Each request takes one token; if the bucket is empty, the request is rejected.

Where to enforce limits

Distributed rate limiting

With many gateway instances, each sees only part of a client's traffic. A per-instance in-memory counter lets a client send N × limit by spreading requests across N instances.

Communicating limits to clients

Good APIs tell clients where they stand:

Design choices around the limiter

Rate limiting vs related techniques

Interview questions

  • Why is the fixed window algorithm problematic, and how do you fix it?
  • How do you make a distributed rate limiter correct under concurrency?
  • Your rate limiter's Redis cluster goes down. What should happen?
  • Token bucket or leaky bucket — when would you choose each?

Practice

Write your answer before opening the suggested one. If you mark it "Partly" or "Missed it", it comes back in your review deck.

Further reading

Key takeaways

  • Rate limiting protects availability, prevents abuse, enforces fairness, and implements pricing tiers.
  • Token bucket allows bursts with an average rate; leaky bucket smooths output; fixed windows are simple but allow boundary bursts; sliding windows fix that.
  • Enforce limits in layers: edge (IP), gateway (user/key), service (business rules).
  • Distributed limiters need a shared store with atomic operations (Redis + Lua) and a clear fail-open/fail-closed policy.
  • Return 429 with Retry-After and rate-limit headers; clients should back off with jitter.

Previous: Probabilistic Data Structures · Next: Availability & Fault Tolerance