Design a Rate Limiter (LLD)
Intermediate · 20 min read · Low-Level Design, Case Studies: Advanced
Pluggable algorithms, per-key buckets and thread safety.
The LLD version of the rate limiter is a favourite machine-coding problem: implement a thread-safe, in-process limiter with pluggable algorithms and per-key, per-rule configuration. (The algorithms are explained in depth in Rate Limiting; the distributed service in Design a Distributed Rate Limiter.)
What the interviewer expects
The full design in this lesson is longer than anything you'd write in one round. It's there so you understand the edge cases and can answer follow-up questions. In the round itself, aim for this:
1. Requirements
2. Class design
3. Implementation
Use a monotonic time source (System::nanoTime) for durations — wall-clock time can jump backwards.
4. Memory: evicting idle keys
Per-key state for millions of users who appear once would leak memory. Options:
5. Patterns used
Interview questions
- How do you make a token bucket thread-safe without a global lock?
- Why use System.nanoTime rather than System.currentTimeMillis for refill calculations?
Further reading
Key takeaways
- One interface, several algorithm implementations (Strategy), built from rule config (Factory).
- State is per key in a ConcurrentHashMap; lock per key (or CAS) for low contention.
- Use a monotonic, injected time source; evict idle keys to bound memory.
- Evaluate multiple rules per request; be explicit about consumption semantics.