18 MINcoreCachingNot started

Cache Aside (Lazy Loading)

Cache aside is the most common caching strategy. The application checks the cache first; on a miss, it fetches from the database, writes to the cache with a TTL, and returns. Writes update the DB and invalidate the cache. It is simple, fault-tolerant, and wastes no memory on unread data — but it allows stale reads and cache stampedes.

Prerequisites

Caching Strategies

Used in

Lesson

How it works

In cache aside (also called lazy loading), the application code manages the cache explicitly. The cache is not pre-populated — it fills lazily based on actual access patterns. Only data that is actually read gets cached.

This is the most common caching strategy because it is simple, fault-tolerant, and memory-efficient. If the cache crashes, the system still works (just slower). If data is never read, it never enters the cache (no wasted memory).

from redis import Redis

cache = Redis(...)

def get_user(user_id):
    key = f'user:{user_id}'

    # 1. Check cache first
    cached = cache.get(key)
    if cached:
        return cached  # cache HIT

    # 2. Cache MISS — fetch from DB
    user = db.fetch_user(user_id)

    # 3. Write to cache with TTL (safety net)
    cache.set(key, user, ttl=300)  # 5 minutes

    return user

def update_user(user_id, data):
    # 1. Update DB first (source of truth)
    db.update_user(user_id, data)
    # 2. Invalidate cache (next read will re-populate)
    cache.delete(f'user:{user_id}')

Cache aside in Python with Redis. Reads check cache, fall through to DB on miss, then populate. Writes invalidate.

Stale data risk

Cache aside trades freshness for speed. Between a write and the cache invalidation, reads can return stale data. TTLs cap the staleness window — a 300s TTL means data is at most 5 minutes stale. For strong freshness, invalidate synchronously on write, but accept that the invalidation can fail (cache can be down) — always set a TTL as a safety net.

Cache stampede (thundering herd):

A subtle failure mode: when a popular key expires, the next N requests all miss simultaneously, all fetch from the database, all write to the cache. The database takes N times the expected load for a brief window.

Example: 10,000 requests/sec for a popular product page. The cache TTL expires. Within the next 100ms, 1,000 requests all see a miss. They all query the database. The database, which was handling 100 queries/sec (the 10% that were misses), now handles 1,000 queries in 100ms — 10x normal load. It may crash or become very slow.

Mitigations:

Cache locking (request coalescing): only the first miss fetches from the DB; others wait for the first to complete and share the result.
Early refresh (refresh-ahead): refresh the cache before it expires, in the background.
Probabilistic early expiration: add jitter to the TTL (e.g., TTL = 300s ± random(0-30s)) so keys don't all expire at the same instant.

When to use cache aside:

Read-heavy workloads (read:write ratio > 10:1).
Data that changes infrequently (product catalog, user profiles).
When you can tolerate brief staleness (social feeds, analytics).
When you want the simplest caching strategy.

When NOT to use cache aside:

When staleness is unacceptable (bank balances, inventory counts). Use write-through instead.
When writes are very frequent and reads are rare. Cache aside adds invalidation overhead on every write for little benefit.
When you need the cache to be the source of truth for writes (write-behind).

Check yourself

solid

Your cache aside system uses a 5-minute TTL on user profiles. A user updates their bio, the DB write succeeds, but the cache invalidation call fails (cache is briefly down). What happens?

Pick one answer.

Check yourself

hard

A popular product page's cache entry expires. Within the same second, 1000 users request the page. What happens without mitigation?

Pick one answer.

The HIT and MISS paths, side by side. Cache aside's apparent simplicity hides two very different code paths the application must execute correctly.

HIT path (cache hit, ~1ms):

App calls cache.get(key).
Redis returns the value in ~0.5-1ms (network + lookup).
App deserializes and returns.

The HIT path is what makes cache-aside fast — it's a single Redis call, no DB involvement, no serialization of complex queries.

MISS path (cache miss, ~50-100ms):

App calls cache.get(key) — returns nil (0.5ms).
App queries the database (10-100ms depending on query complexity).
App serializes the result and calls cache.set(key, value, TTL) (1ms).
App returns the value.

The MISS path is what makes cache-aside slow on the first request — but it's a one-time cost amortized over many subsequent HITs. The ratio of HITs to MISSes is the cache hit rate — typically 90-99% in production. At 95% hit rate, average latency = 0.95 × 1ms + 0.05 × 50ms = 3.45ms — far below the un-cached 50ms.

The subtle bug: if step 3 (cache.set) fails silently (Redis momentarily down, network blip), the next read will also miss — and the next, and the next. The cache stays empty until Redis recovers. Most production code logs and alarms on set-failure rates to catch this.

Write invalidation — the order matters. When the application updates the database, it must also update (or invalidate) the cache. The order of operations determines which failure modes you're exposed to:

Pattern 1: Update DB, then update cache. If the cache update fails, the cache now holds stale data until TTL expires. Worst case: a write to the DB never propagates to the cache, and reads return the old value for the full TTL.

Pattern 2: Update DB, then invalidate cache (delete). On the next read, the cache miss repopulates from the DB. If the delete fails, the cache holds stale data until TTL expires — same staleness window as Pattern 1, but the cache is empty (not stale) for the brief moment between DB update and delete.

Pattern 3 (BAD): Update cache, then update DB. If the DB write fails after the cache was updated, the cache now holds data that doesn't exist in the DB. Reads will return 'successful' results for an operation that didn't actually happen. Never use this pattern.

Pattern 4 (BAD): Delete cache, then update DB. Between the delete and the DB update, another read can MISS, fetch the OLD value from the DB, and write the OLD value back to the cache — overwriting the soon-to-be-updated value. The cache and DB diverge, and the cache stays stale until TTL expires. This is a classic race condition.

The consensus: use Pattern 2 (update DB first, then invalidate cache), and always set a TTL as a safety net for failed invalidations. The TTL bounds the maximum staleness window — even if invalidation fails, the cache will eventually expire and re-fetch from the DB.

Cache stampede mitigations, compared. All three approaches reduce the thundering-herd problem; each has different trade-offs.

Mitigation	How it works	Pros	Cons
Cache lock	First MISS acquires a lock; others wait.	Simple, DB sees 1 query.	Waiters pay extra latency (up to the DB query time). Lock holder is a single point of failure.
Probabilistic early expiration	TTL = base + random(-jitter, +jitter). Misses spread over time instead of clustering.	No coordination needed; decentralized.	Some misses still happen, just spread out. Tuning the jitter is tricky.
Refresh-ahead	Background job refreshes popular items before expiry.	Users never see a miss for popular items.	Wastes work refreshing items that may not be read.

The production choice depends on the workload:

Hot key with predictable traffic (home page, top product): refresh-ahead — zero misses, low latency, worth the wasted refresh.
Many keys with spiky access (user profiles): probabilistic early expiration — cheap, scales, requires no coordination.
One-off expensive query (rare but slow): cache lock — keep the DB safe, accept the waiter latency.

In practice, most teams combine: refresh-ahead for the top-N hottest keys, probabilistic jitter for the long tail, and cache locking as the last-resort safety net on the rarest, most expensive queries. Netflix, Facebook, and Twitter all use combinations of these techniques.

import redis
import json
import time
import uuid

r = redis.Redis(...)

# Cache-aside with stampede protection via SET NX (lock)

def get_user(user_id):
    key = f'user:{user_id}'
    lock_key = f'lock:{key}'

    # 1. Fast path: cache HIT
    cached = r.get(key)
    if cached:
        return json.loads(cached)

    # 2. MISS - try to acquire a lock with NX (only one wins)
    #    Others wait and retry the cache read.
    lock_token = str(uuid.uuid4())
    acquired = r.set(lock_key, lock_token, nx=True, ex=10)  # 10s TTL on the lock

    if not acquired:
        # Another request is fetching - wait and retry cache
        for _ in range(20):
            time.sleep(0.05)  # 50ms
            cached = r.get(key)
            if cached:
                return json.loads(cached)
        # Lock held too long - fall through to DB as a safety net

    try:
        # 3. Cache MISS path: fetch from DB (the slow part)
        user = db.fetch_user(user_id)

        # 4. Populate cache with TTL (safety net for failed invalidation)
        #    Add +/- 30s jitter to spread out future expiries (stampede mitigation)
        jitter = random.randint(-30, 30)
        r.set(key, json.dumps(user), ex=300 + jitter)
        return user
    finally:
        # 5. Release the lock - only if we still hold it (use Lua for atomic check-and-del)
        r.eval(
            'if redis.call("get", KEYS[1]) == ARGV[1] then '
            '  return redis.call("del", KEYS[1]) '
            'else return 0 end',
            1, lock_key, lock_token
        )

def update_user(user_id, data):
    # Pattern 2: update DB first, then invalidate cache
    db.update_user(user_id, data)
    r.delete(f'user:{user_id}')  # if this fails, TTL is the safety net

Production cache-aside in Python with stampede protection (cache lock + TTL jitter) and safe write invalidation (DB first, then delete).

Key design matters as much as TTL

A subtle cache-aside bug: cache key choice. If you cache by user_id only, all reads of a user share one entry — fine. But if you cache user:42:with_orders (a join of user + their orders) under the same user:42 key as the basic profile, an order update invalidates the basic profile too — unnecessary. Use specific keys per shape of data: user:{id}, user:{id}:with_orders, user:{id}:permissions. On write, invalidate only the keys whose shape changed. Netflix's EVCache uses a versioned key scheme: user:{id}:v{schema_version} — bumping the version invalidates the whole shape atomically. The lesson: cache key design is a schema decision, not an afterthought. Plan your cache keys like you plan your database schema.

Check yourself

interview

Your team implements cache invalidation as: `cache.delete(key); db.update(...)`. The code review flags it as a race condition. Why?

Pick one answer.

Try this

interview

The cache.delete call appears in your logs as successful. Redis is healthy. The DB write is committed.

A user updates their profile bio. They refresh the page 2 seconds later and still see the old bio. They refresh again 10 seconds later — old bio. They refresh at 60 seconds — finally the new bio appears. Your code does: `db.update(...); cache.delete(key)` with TTL=300s. Diagnose.

Engineering mental model

Mental model. Think of Cache Aside (Lazy Loading) as a deliberate boundary in a system. The boundary exists because something becomes harder to manage when everything is done in one place: latency, scale, failure isolation, consistency, cost, or team ownership. The useful question is not “what does Cache Aside (Lazy Loading) mean?” but “what pressure makes this boundary worth introducing, and what new failure mode does it create?”

Design lens

Before choosing Cache Aside (Lazy Loading), name the workload, the critical user path, the dominant bottleneck, the failure you are trying to absorb, and the trade-off you are willing to accept. If you cannot name those five things, the design is probably premature.

Cache Aside (Lazy Loading): a compact system-thinking visual.— Original NO CAP visual.

value = cache.get(key)
if value is None:
    value = load_from_origin(key)
    cache.set(key, value, ttl=300)
return value

A minimal engineering sketch for reasoning about Cache Aside (Lazy Loading).

Back-of-the-envelope reasoning

Example: 100,000 requests/s at 90% cache hit rate means roughly 10,000 requests/s reach the origin. Raising the hit rate from 90% to 95% cuts origin traffic in half again.

Interactive sandboxdeterministic

Interactive thought experiment: Cache Aside (Lazy Loading)

Change the variables below and predict what breaks first in Cache Aside (Lazy Loading). The production lab can later reuse these same inputs.

request_rate10,000

110000

Requests per second

hit_rate1

Cache hit percentage

ttl300

010000

Cache freshness window

System pressure46%

Approx. origin requests1,000/s

Try this

Change one variable at a time. Predict the failure mode first, then move the slider and see whether your mental model matches the simplified system response.

Hint

If you are stuck on Cache Aside (Lazy Loading), start by drawing the request path and marking every network hop, stateful component, queue, cache and failure boundary. Then estimate where the system will saturate.

Check yourself

solid

You increase traffic by 10× in a system using Cache Aside (Lazy Loading). What should you inspect first?

Pick one answer.

Check yourself

interview

Which statement is the safest engineering habit when using Cache Aside (Lazy Loading)?

Pick one answer.

Try this

interview

You have dashboards for traffic, latency, errors and saturation. You can change the architecture, but every change has operational cost.

Production scenario: your system uses Cache Aside (Lazy Loading), traffic suddenly spikes, and p99 latency doubles. What is your first move?

Interview drill

Answer this without notes: When would you choose Cache Aside (Lazy Loading), and when would you intentionally avoid it? Mention at least one bottleneck it addresses, one failure mode it introduces, and one alternative. Then quantify the workload you are designing for.

Engineering lens

For Cache Aside (Lazy Loading), reason in this order: what data is hottest, where can it live closer to the caller, what makes it stale, and what happens on a miss or cache failure. A cache is an optimization boundary, not the source of truth.

Numerical sanity check

A useful first-order model is cache_load = request_rate × (1 - hit_rate). If traffic is 20,000 req/s and the hit rate is 90%, roughly 2,000 req/s still reaches the origin before considering misses caused by expiration or eviction.

Check yourself

interview

Do not optimize for a memorized definition. Reason from the workload and failure mode.

You have a read-heavy Cache Aside (Lazy Loading) path. Traffic doubles overnight. What metric would you inspect first, and what would convince you to add another cache layer?

Pick one answer.

Trade-offs

What you gain, what you pay

Pros

+Lazy population — cache only holds data someone actually reads.
+Fault-tolerant — if cache dies, system keeps working (just slower).
+Simple to implement — GET, SET, DEL.
+Memory-efficient — no wasted entries.

Cons

−Stale data window up to TTL duration.
−Cache stampede on popular key expiry.
−Application code is aware of the cache (not transparent).
−First read after expiry is slow (cache miss).

Failure modes

How this breaks in production

Stale data — invalidation fails, TTL hasn't expired yet.
Cache stampede — popular key expiry causes thundering herd.
Cache thrashing — keys evicted before they're read again (cache too small).
Inconsistent state — DB updated but cache invalidation failed.

Common mistakes

Don't fall into these traps

•No TTL. Without a TTL, a cache invalidation failure means stale data forever.
•Updating cache before DB. If the cache write succeeds but the DB write fails, you are now inconsistent. Always write DB first, then invalidate cache.
•Forgetting cache stampede protection on hot keys.
•Caching everything. Caching rarely-read data wastes memory and adds invalidation complexity.

Where you see it

Real systems using this

Netflix, web apps, APIs — everywhere reads outnumber writes.Redis, Memcached, Varnish, CDN edge caching.Browser cache, OS page cache — caching is everywhere, layered.

Teardowns

How real systems implement this

Netflix — Multi-tier caching: CDN edge → EVCache (origin cache, cache-aside) → database. 90%+ of reads served from cache. Cache-aside for content metadata.
Redis — The most popular in-memory cache. Used in cache-aside mode by most applications: GET on read, SET on miss, DEL on write.

Interview prompts

Practice saying it out loud

Q1Design a caching layer for a news feed that gets 10k reads/sec and 10 writes/sec.
Q2How do you handle cache stampedes? What are the trade-offs of each approach?
Q3Your cache and database diverged. How do you detect it, and how do you fix it?
Q4When would you NOT use cache aside? What are the alternatives?

Research

Cache Aside (Lazy Loading)

How it works

Engineering mental model

Back-of-the-envelope reasoning

Interactive thought experiment: Cache Aside (Lazy Loading)

Interview drill

Numerical sanity check

What you gain, what you pay

How this breaks in production

Don't fall into these traps

Real systems using this

How real systems implement this

Practice saying it out loud

Further reading & references

Cache Aside (Lazy Loading)

How it works

Engineering mental model

Back-of-the-envelope reasoning

Interactive thought experiment: Cache Aside (Lazy Loading)

Interview drill

Numerical sanity check

What you gain, what you pay

How this breaks in production

Don't fall into these traps

Real systems using this

How real systems implement this

Practice saying it out loud

Further reading & references