Offered by Dead Brands
How to put fast, disposable copies of data between callers and slow sources of truth: the read/write patterns (cache-aside, write-through, write-behind), TTL design, eviction policies (LRU/LFU), CDN l
Use it over MCP: connect to https://mcp.deadbrands.co/hub/mcp and call skills_get with slug caching-strategies.
# Caching Strategies Credit: SteTy — published free in the Skills Hub How to put fast, disposable copies of data between callers and slow sources of truth: the read/write patterns (cache-aside, write-through, write-behind), TTL design, eviction policies (LRU/LFU), CDN layers, and the defenses — coalescing, jitter, negative caching — that keep a cache from stampeding the very database it protects. ## Overview A cache stores the result of an expensive operation (a query, a computation, an HTTP response) so the next request skips the work. The economics are stark: in-process memory answers in nanoseconds, a network cache in ~0.5ms, a database query in 5–500ms. But a cache is a *copy*, and every copy can be stale, wrong, or missing at the worst moment. As Phil Karlton's famous line puts it, cache invalidation is one of the two hard problems in computer science. This skill is the standard playbook for getting the speed without the classic failures. ## Core Principles - **The database is the truth; the cache is a rumor.** Any design must survive the cache being emptied at any instant — flushed, restarted, evicted — with correctness intact and only speed lost. - **Cache what's expensive, read-often, and slow-changing.** Caching data that's cheap to compute, rarely read, or changing every second buys complexity and returns stale answers. - **Measure the hit ratio or you're guessing.** Below ~80% hits, the strategy needs review; a cache absorbing 95%+ of reads is doing its job. Also track hit ratio *per key class* — one hot key class can be failing while the average looks fine. - **Staleness is a budget, not a bug.** Every cache serves data up to TTL-old unless explicitly invalidated. Decide per dataset how stale is acceptable, then pick the mechanism that enforces it. - **Layers, not one cache.** Browser → CDN edge → application in-process cache → distributed cache → database. Each layer absorbs load for the next; each has its own TTL and consistency story. ## Techniques **Read patterns.** - **Cache-aside (lazy loading)** — the default. Application checks the cache; on a miss it reads the database, writes the value into the cache with a TTL, and returns it. On write, update the database and **delete** the cache key (don't update it — deletion avoids the race where a concurrent stale read repopulates the old value after your update). Only data actually requested gets cached, and a dead cache degrades to direct database reads instead of an outage. - **Read-through** — the cache library itself loads from the source on a miss. Same behavior, but the loading logic is centralized instead of copied through the codebase. **Write patterns.** - **Write-through**: writes update cache and database synchronously. Read-after-write is consistent; writes pay both latencies. Right for data read back immediately after writing. - **Write-behind (write-back)**: write to the cache, acknowledge, flush to the database asynchronously. Fastest writes and coalesces repeated updates into one — but a cache crash before flush **loses data**. Fit: counters, view counts, metrics. Never: orders, payments. - **Write-around**: writes go straight to the database, bypassing the cache; reads populate it later. Keeps one-shot writes from evicting useful entries. **TTL design.** TTL is the staleness bound and the backstop for every missed invalidation — always set one, even with event-driven invalidation. Typical starting points: static config/reference data ~24h; user profiles ~5 min; product catalog 1–5 min; trending feeds 30–60s; live pricing/inventory 5–15s; session entries match session lifetime. Always add **jitter**: `TTL = base + random(0, 10% of base)` so thousands of keys written together don't expire together. **Eviction.** When memory fills, something must go. **LRU** (least recently used) is the safe default — recency predicts reuse. **LFU** (least frequently used) wins when popularity is stable over time (top products, hot content); it needs aging so a formerly-hot key can't squat forever. **FIFO** ignores usage entirely — simple, rarely best. Redis-style `volatile-*` variants evict only among keys that have a TTL, protecting keys that must never vanish. Eviction is not expiration: TTL bounds staleness, eviction bounds memory, and you usually need both. **CDN layers.** A CDN caches HTTP responses at edge locations near users. Control it with headers: `Cache-Control: max-age` for freshness, `s-maxage` for shared caches, `ETag`/`Last-Modified` for cheap revalidation, and `stale-while-revalidate` to serve the stored copy while the edge refreshes it in the background. Invalidate by purging URLs or — cleaner — by **versioning the URL** (`/app.v42.js`), so new content is a new key and old copies die by TTL. Never cache personalized or authenticated responses in a shared layer without a correct `Vary` and cache key. **Stampede (thundering herd) protection.** The failure mode: a key serving 10,000 req/s expires, and hundreds of simultaneous misses all recompute at once — Facebook's memcached fleet famously saw database load spike from ~1,300 to ~17,000 queries/s on a single expiry. Defenses, combined: - **Request coalescing / single-flight**: the first misser takes a lock (e.g., `SET NX` with a short expiry) and recomputes; everyone else waits and reads the fresh value. N misses become 1 database call. Facebook's lease mechanism cut peak database queries ~92%. - **Probabilistic early refresh**: as a key nears expiry, each request has a growing chance of refreshing it early while others still serve the current value — the key never actually expires under load. - **Stale-while-revalidate**: serve the slightly stale value while one background refresh runs. - **Warm the cache** before traffic shifts (deploys, restarts, known peaks) instead of letting the first wave of users pay the misses. **Invalidation.** Delete-on-write is the workhorse: write to the database, delete affected keys, next read repopulates. For derived data (lists, aggregates, search results), one write may fan out to many keys — versioned keys (`user:42:v7`) make invalidation O(1): bump the version, old keys orphan and expire. Multi-service systems use event-driven invalidation or change-data-capture streams. Every approach still keeps a TTL backstop, because invalidations get missed. **Negative caching.** Cache the *absence* of a result too: "user 999 does not exist" for 30–60s. Without it, repeated lookups for nonexistent keys (typos, scanners, broken clients) punch straight through to the database every time — cache penetration. Short TTL, because the thing may legitimately appear soon. **Key design.** Keys are a namespace contract: `entity:id:field`, include a schema/version component, and never build keys from unbounded user input without hashing — key cardinality explosions eat memory. ## Worked Examples 1. **Product page, cache-aside.** Read `product:881` → miss → query database (40ms) → cache with TTL 240s + jitter → subsequent reads ~0.5ms. Price update writes the database and deletes `product:881`; the next read repopulates with the new price. Worst-case staleness: bounded by TTL if a delete is ever missed. 2. **Homepage stampede, prevented.** A homepage key at 8,000 req/s uses single-flight plus early refresh: the value is recomputed by exactly one request roughly every TTL window, in the background of normal traffic. The database sees ~1 query per window instead of a 500-request pile-up at expiry. 3. **Phantom product.** A deleted product keeps being requested by an old app version. Without negative caching: 2,000 database queries/min confirming nonexistence. With a 30s negative entry: ~2 queries/min, and the product reappearing is picked up within half a minute. ## Common Pitfalls - Caching before measuring: adding a cache to a 2ms query and inheriting every invalidation bug for nothing. - Updating the cache on write instead of deleting — invites the read-modify race that resurrects stale values. - No TTL "because we invalidate properly." You don't. Everyone misses one path. - Identical TTLs on bulk-loaded keys → synchronized mass expiry (an avalanche) on a timer. - Caching errors and empty responses unintentionally for the full TTL. - Treating eviction as a strategy: if hot data is being evicted, the fix is memory or key design, not a different policy. - Cache keys that omit a dimension (tenant, locale, version) — serving one user's or language's data to another. ## Practice Drills 1. Pick a dataset and defend a TTL from its change rate and staleness cost; then pick the write pattern and say what breaks if the cache dies mid-write. 2. Simulate a stampede: expire a hot key under load with and without single-flight; count database queries in each run. 3. Audit a keyspace: find the keys with no TTL, the keys with identical bulk TTLs, and the three largest values. 4. Design the invalidation fan-out for "user renames themselves" across profile, feed, and search caches. ## When to Use and Limits - **Use it when:** a task calls for caching strategies, as laid out in the Techniques and Worked Examples above. - **Watch first for:** (see Common Pitfalls). - **Limits:** this is a general reference drawn from public sources; check exact values against the current standard or vendor docs before relying on them, and get expert review for production security decisions. ## Quick Reference - Default stack: cache-aside + delete-on-write + TTL backstop with jitter. - Write-through = consistent, slower writes. Write-behind = fast writes, loss risk. Write-around = don't pollute. - LRU default; LFU for stable popularity; TTL always. - Stampede kit: single-flight lock, early refresh, stale-while-revalidate, warming. - If it doesn't exist, cache that too — briefly.