Caching
Cache-aside, invalidation, stampedes, eviction
The problem
Databases live on disk and cost real time per read; recomputing the same answers wastes resources. Caching is always the same move at every layer: a small fast memory sits in front of a big slow storage, holding copies of what’s hot. Browser cache, DNS cache, CDN edge, Redis in front of your database — one idea, four costumes. The health metric is hit rate: at 95%, your database only feels 5% of the traffic.
Cache-aside — the default pattern
Check the cache: hit → return it. Miss → fetch from the database, return it, and leave a copy in the cache so the next request hits. The cache fills lazily with what’s actually requested — the same pull-through idea CDNs use.
Invalidation — the famous hard problem
When the source data changes, caches everywhere still hold the old copy — and the hotter the data, the longer it lives, because hot entries never miss. Three families of fix: TTL (every entry expires after N seconds — dumb, reliable, staleness is bounded), event-based invalidation (the write path deletes the cache entry — fresh, but every write path must remember, and one forgotten path poisons caches silently), and versioned keys (keys change on update, so stale entries just stop being requested). TTL buys simplicity at the price of bounded staleness; event-based buys freshness at the price of complexity. Sixty seconds of stale profile photo is fine; sixty seconds of stale bank balance is not.
Write patterns
Write-through: writes go to cache and database together — never stale, every write pays double. Write-behind: write to the cache only, confirm instantly, sync to the database later — fastest writes, but the cache is volatile memory: crash before the sync and the data is gone after the user was told "saved." A cache is an accelerator, not a source of truth — write-behind is for data you can afford to lose a sliver of (counters, analytics), never orders or money.
The stampede
A hot key’s TTL expires and a thousand concurrent requests all miss at once — a thousand identical queries hit a database sized for 5% of traffic. Fixes: TTL jitter (randomize expiries so hot keys never die in sync), single-flight locks (first miss fetches, the rest wait or get the just-expired copy), early refresh (renew hot keys before they expire), and hot-key replication (spread copies of celebrity-grade keys across nodes).
Eviction and dying gracefully
Caches are small on purpose, so something must be kicked when they fill. The default is LRU — evict the least recently used — because recency tracks what’s hot now, while LFU (least frequently used) worships the past: one formerly-viral entry can squat forever on old hit counts. When the whole cache dies, 100% of traffic hits a database sized for a fraction of it: mitigate with cache replication and circuit breakers that shed load rather than letting everything collapse. Tools: default to Redis (rich structures, persistence, pub/sub); Memcached if all you need is dumb fast key-value.