Caching Strategies

A cache is a faster, smaller storage layer that sits between the application and the slower, larger source of truth (usually a database). The application checks the cache first; on a hit, it returns the cached value (fast). On a miss, it fetches from the database (slow) and writes the result to the cache for next time.

The caching strategy determines when and how data is written to the cache. There are four main strategies, each with different trade-offs in freshness, write latency, and failure behavior.

Cache Aside (Lazy Loading) — the most common strategy.

The application manages the cache explicitly:

1. Check cache for the key.
2. If hit → return the cached value.
3. If miss → fetch from DB, write to cache with TTL, return.

Writes update the database, then invalidate (delete) the cache entry. The next read will miss and re-populate from the DB.

Pros:

• Cache only holds data that's actually read (no wasted memory).
• Fault-tolerant: if the cache crashes, the system still works (just slower).
• Simple to implement.

Cons:

• Stale data: between a write and the cache invalidation, reads can return old data. TTLs cap the staleness window.
• Cache stampede: when a popular key expires, N concurrent requests all miss and all fetch from the DB simultaneously.
• Application code is aware of the cache (not transparent).

Write Through — the cache is always fresh.

Writes go to both the cache AND the database synchronously. The application writes to the cache; the cache writes to the DB and returns only after both succeed. Reads always hit the cache.

Pros:

• Cache is always fresh — no stale reads.
• Read performance is optimal (always cache hit, assuming data has been written).
• Simple read path (no DB fallback).

Cons:

• Higher write latency (must write to cache + DB synchronously).
• If the cache is down, writes fail (unless you add fallback logic).
• Cache holds everything ever written, even if never read (wasted memory).

Best for: data that's written once and read many times, where staleness is unacceptable.

Write Behind (Write-Back) — the fastest writes, but risky.

Writes go to the cache only. The cache returns immediately. A background process writes to the DB asynchronously (batched, delayed).

Pros:

• Extremely fast writes (only one write to cache, no DB round-trip).
• DB load is smoothed (batched writes).
• Survives brief DB outages (writes queue in cache).

Cons:

• Data loss risk: if the cache crashes before flushing to DB, committed writes are lost.
• Reads from the DB (by other services, analytics) can see stale data.
• Complex to implement (write queue, flush logic, ordering).

Best for: high-write, low-criticality data (counters, analytics events, telemetry). Never use for payments or transactions.

Refresh Ahead — popular items never expire.

The cache proactively refreshes entries before their TTL expires. A background process watches for entries nearing expiry; if they're frequently accessed, it refreshes them from the DB before they expire.

Pros:

• Popular items never miss — the user never waits for a DB fetch.
• Smooths DB load (refreshes are spread over time, not bursty).

Cons:

• Wastes resources refreshing items that nobody reads.
• Complex to implement (need to track access frequency).
• If the refresh fails, the item expires normally.

Best for: systems with clear 'hot' items (home page content, popular product pages, trending topics).

Cache hierarchy — layered caches compose. Production systems rarely have one cache. They have a stack: each layer is faster but smaller, and each serves misses from the layer below.

A typical web app stack:

1. Browser cache (per-user, seconds-to-days TTL). Static assets with hashes in filenames (e.g., app.abc123.js) cached for a year. HTML cached for seconds.
2. CDN edge cache (per-PoP, minutes-to-hours TTL). Static assets, sometimes HTML, cached at 300+ PoPs globally. ~10ms latency to user.
3. App-server in-process cache (per-instance, seconds TTL). Frequently accessed config, computed values. ~0.01ms latency. Lost on restart.
4. Distributed cache (Redis/Memcached) (shared, seconds-to-minutes TTL). User profiles, session data, query results. ~1ms latency. Survives app restarts.
5. Database query cache (per-DB, often disabled). Cached query results. Often more trouble than it's worth (invalidation is hard).
6. Database buffer pool (per-DB, transparent). Pages cached in RAM. ~0.1ms latency. The OS page cache provides another layer.

The key insight: each layer only holds a fraction of the data, and each layer serves ~90% of its requests from cache. So 1000 user requests might result in 100 CDN misses (10%), of which 10 reach the app (90% served from CDN), of which 1 reaches the DB (90% served from Redis). The DB sees 0.1% of original load. This is why a small cache can carry a 1000x-larger database.

All four strategies, side by side.

The right strategy depends on (1) read:write ratio, (2) staleness tolerance, (3) write latency tolerance, and (4) data-loss tolerance. Most systems combine: cache-aside as the default, write-through for transactional state, write-behind for counters and analytics, refresh-ahead for hot keys.

The architect's rule: start with cache-aside. Add complexity only when measurements prove you need it. Most cache-related outages come from over-engineering (write-behind on data you can't afford to lose) or under-engineering (cache-aside on a hot key without stampede protection).

Mental model. Think of Caching Strategies as a deliberate boundary in a system. The boundary exists because something becomes harder to manage when everything is done in one place: latency, scale, failure isolation, consistency, cost, or team ownership. The useful question is not “what does Caching Strategies mean?” but “what pressure makes this boundary worth introducing, and what new failure mode does it create?”

Example: 10M requests/day ÷ 86,400 ≈ 116 requests/s average. Design for peak rather than average; a 10× peak is ≈ 1,160 requests/s.

Answer this without notes: When would you choose Caching Strategies, and when would you intentionally avoid it? Mention at least one bottleneck it addresses, one failure mode it introduces, and one alternative. Then quantify the workload you are designing for.

Numerical sanity check

A useful first-order model is cache_load = request_rate × (1 - hit_rate). If traffic is 20,000 req/s and the hit rate is 90%, roughly 2,000 req/s still reaches the origin before considering misses caused by expiration or eviction.