Caching

Intermediate · 20 min read · High-Level Design, Caching & Content Delivery

Cache layers, read/write strategies, eviction, invalidation, and stampedes.

A cache stores a copy of data in a faster location so future requests can be served without going back to the slower source. Caching is the single most effective technique for improving read latency and reducing load on databases — and also one of the most common sources of subtle bugs.

Why caching works

Two properties of real workloads make caching effective:

Where caches live

A request may pass through many cache layers:

Cache read/write strategies

The application manages the cache explicitly. This is the most common pattern.

Eviction policies

Caches have finite memory. When full, the eviction policy decides what to drop.

Cache invalidation

Keeping cached data in sync with the source of truth is the hard part. Options, from simplest to most precise:

Failure modes every design should address

A popular key expires; thousands of concurrent requests miss simultaneously and all hit the database to recompute it.

Distributed caching: Redis vs Memcached

What to cache — and what not to

Good candidates: data that's read far more than written; expensive to compute (aggregations, rendered pages, ML features); tolerant of brief staleness; or accessed with a skewed distribution.

Interview questions

  • Your cache hit ratio is 95% but the database is still overloaded. What do you check?
  • Why delete the cache entry on update instead of writing the new value?
  • How do you prevent a cache stampede on a very popular key?
  • When would you choose an in-process cache over Redis?

Practice

Write your answer before opening the suggested one. Anything you mark "Partly" or "Missed it" comes back in your review deck.

Further reading

Key takeaways

  • Caching exploits skewed access and the memory/disk speed gap; the hit ratio determines its value.
  • Cache-aside with delete-on-write and a TTL safety net is the default pattern.
  • Write-through keeps cached data fresh; write-behind is fastest but risks data loss.
  • LRU is a solid default eviction policy; TTLs bound staleness.
  • Plan for stampedes, penetration, avalanches and hot keys — and make sure the system degrades gracefully if the cache fails.

Previous: Unique ID Generation · Next: Content Delivery Networks