Application Caching
Application caching stores computed results, objects, or rendered fragments in an in-memory store (Redis, Memcached) accessed by application code. It is the most common cache layer — the one engineers think of when they say "add a cache." Application caching is explicit, per-object, and flexible: the application decides what to cache, how to compute the key, and what TTL to set. The cost is cache coherence complexity, the cache-as-single-point-of-failure risk, and the operational burden of running a separate distributed store.
Foundational.
How it works
Application caching is the cache layer that lives in (or alongside) the application tier. The application explicitly reads from and writes to an in-memory store — typically Redis or Memcached — to cache the result of expensive operations: database queries, API calls, computed aggregations, rendered templates. It sits between the web-server cache and the database cache in the standard hierarchy:
client cache → CDN → web-server cache → APPLICATION CACHE → DB cache → DBThe defining property of application caching is that the application code is aware of the cache. Unlike the database buffer pool (transparent to the app) or the OS page cache (transparent to everything), the application explicitly calls cache.get(key), decides what to put there, sets the TTL, and handles invalidation. This makes it the most flexible cache layer — but also the one with the most engineering surface area for bugs.
The two dominant tools are Redis and Memcached, with very different design philosophies:
- Redis: single-threaded (mostly), persistent (optional), rich data types (sorted sets, HyperLogLog, streams, bitmaps), supports Lua scripting, pub/sub, and transactions. The default choice for most modern applications.
- Memcached: multi-threaded, memory-only (no persistence), pure key-value (strings only), simpler and slightly faster for pure KV workloads. Still popular at massive scale (Facebook, Twitter) where its simplicity is an advantage.
The choice rarely matters at small scale. At large scale, the differences matter enormously — Redis's data types enable patterns (sorted-set leaderboards, HyperLogLog cardinality) that Memcached cannot do, while Memcached's multi-threading scales across cores in ways Redis (single-threaded for command execution) cannot match.
What gets cached — the four common patterns.
-
Database query result caching. Cache the result of
SELECT * FROM users WHERE id = 42under the keyuser:42. The most common pattern. Cache-aside is the default strategy: GET, on miss fall through, SET with TTL. -
Object / entity caching. Cache the deserialized object, not the raw DB row. A
Userobject might require joining 3 tables; cache the assembled object instead of re-joining on every read. The cache key is the entity ID; the value is the serialized object (JSON, MessagePack, protobuf). -
Computed / aggregated value caching. Cache the result of expensive computations: "unread notification count for user 42," "top 10 trending posts," "recommendations for user 42." These can take hundreds of ms to compute but rarely change; caching reduces them to 1 ms.
-
Rendered fragment caching. Cache the rendered HTML or JSON fragment: "the sidebar for a logged-in user." Skips both the database work AND the template rendering. Particularly common in Rails (Russian doll caching) and server-rendered React.
The deeper insight is that the closer to the response you cache, the more work you eliminate. Caching the database row eliminates the DB call but not the rendering. Caching the rendered fragment eliminates everything. Caching the entire HTTP response (web-server cache) eliminates even the application's request handling. Each layer outward caches a larger unit of work — and is correspondingly harder to invalidate.
Choose Redis if you need: persistence (durability on restart), rich data types (sorted sets for leaderboards, HyperLogLog for cardinality), pub/sub, Lua scripts, or atomic operations beyond GET/SET. Choose Memcached if: you need maximum raw throughput on a multi-core machine (Memcached is multi-threaded; Redis is single-threaded for command execution), you want the simplest possible model (pure KV strings), or you are operating at Facebook/Twitter scale where Redis's single-threaded bottleneck matters. For most applications, Redis is the default. Memcached is still common at extreme scale and in legacy deployments.
The cache-as-SPOF problem.
When the application cache becomes a system's primary read path, it also becomes a single point of failure. If Redis goes down, every cache miss falls through to the database — and at the hit rates typical of production (90%+), that means the database suddenly sees 10x its normal load. The database, sized for cached traffic, collapses.
This is the most dangerous failure mode of application caching: the cache protects the database, and when the cache fails, the database dies. The standard mitigations:
- Cache in HA mode — Redis Sentinel or Redis Cluster for automatic failover. Memcached does not have native HA; you run multiple instances and accept partial cache loss.
- Circuit breaker on cache miss. If the cache is down, do not let every request fall through to the DB. Shed load (return errors or stale defaults) until the cache recovers.
- Overprovision the database for the un-cached load. Expensive but safe — the DB must handle 100% of read traffic if the cache fails. Most systems do NOT do this; they accept that a cache failure means degradation.
- Graceful degradation. When the cache is down, serve reduced results (empty sidebars, cached snapshots from a secondary cache).
- Cache warmup after restart. A fresh Redis has an empty cache; the database takes the full hit until the cache warms up. Pre-populate critical keys before opening traffic.
The deeper lesson: a cache is a load-shedding layer, not a durability layer. Design your system to survive the cache being down — even if degraded — because it will be down at some point, usually during an incident.
When to use application caching:
- Read-heavy workloads (read:write ratio > 10:1). The cache hit rate will be high and the cost amortizes.
- Expensive queries (joins, aggregations, full-text search) that are repeated.
- Session storage — Redis is the canonical session store for web apps.
- Rate limiting and counters — atomic INCR is perfect for request counting.
- Distributed coordination — locks, leader election, coordination via pub/sub.
When NOT to use application caching:
- Write-heavy workloads — the cache adds write invalidation cost without reducing read load.
- When the working set is larger than RAM — cache hit rate collapses, and you pay the cache maintenance cost for no benefit.
- For data that must be perfectly fresh (bank balances) without a write-through strategy.
- As a substitute for fixing slow queries — the cache hides the problem but does not solve it. A slow query that is cached 90% of the time is still slow 10% of the time.
- For low-traffic internal services where the database can handle the load directly. Caching adds operational complexity that low-traffic services do not need.
Your application uses Redis as a cache with a 95% hit rate. The database is sized to handle the 5% miss traffic. Redis crashes. What happens next, and why is this dangerous?
Pick one answer.
You need to maintain a real-time leaderboard of the top 100 players by score, with millions of score updates per second. Which tool and data structure do you use, and why?
Pick one answer.
Your application caches rendered HTML fragments (sidebars, cards) instead of database rows. What is the trade-off versus caching just the DB rows, and when is fragment caching the right call?
Pick one answer.
Engineering mental model
Mental model. Think of Application Caching as a deliberate boundary in a system. The boundary exists because something becomes harder to manage when everything is done in one place: latency, scale, failure isolation, consistency, cost, or team ownership. The useful question is not “what does Application Caching mean?” but “what pressure makes this boundary worth introducing, and what new failure mode does it create?”
Before choosing Application Caching, name the workload, the critical user path, the dominant bottleneck, the failure you are trying to absorb, and the trade-off you are willing to accept. If you cannot name those five things, the design is probably premature.
// Pseudocode
request = receive()
result = application_caching(request)
return result
// Production questions:
// 1. What happens on timeout?
// 2. Can this operation be retried safely?
// 3. What is the bottleneck?Back-of-the-envelope reasoning
Numerical lens: write down traffic, payload size, read/write ratio, peak multiplier and durability target before choosing a component. The numbers should justify the architecture.
Interactive thought experiment: Application Caching
Change the variables below and predict what breaks first in Application Caching. The production lab can later reuse these same inputs.
Change one variable at a time. Predict the failure mode first, then move the slider and see whether your mental model matches the simplified system response.
If you are stuck on Application Caching, start by drawing the request path and marking every network hop, stateful component, queue, cache and failure boundary. Then estimate where the system will saturate.
You increase traffic by 10× in a system using Application Caching. What should you inspect first?
Pick one answer.
Which statement is the safest engineering habit when using Application Caching?
Pick one answer.
You have dashboards for traffic, latency, errors and saturation. You can change the architecture, but every change has operational cost.
Production scenario: your system uses Application Caching, traffic suddenly spikes, and p99 latency doubles. What is your first move?
Interview drill
Answer this without notes: When would you choose Application Caching, and when would you intentionally avoid it? Mention at least one bottleneck it addresses, one failure mode it introduces, and one alternative. Then quantify the workload you are designing for.
For Application Caching, reason in this order: what data is hottest, where can it live closer to the caller, what makes it stale, and what happens on a miss or cache failure. A cache is an optimization boundary, not the source of truth.
Numerical sanity check
A useful first-order model is cache_load = request_rate × (1 - hit_rate). If traffic is 20,000 req/s and the hit rate is 90%, roughly 2,000 req/s still reaches the origin before considering misses caused by expiration or eviction.
Do not optimize for a memorized definition. Reason from the workload and failure mode.
You have a read-heavy Application Caching path. Traffic doubles overnight. What metric would you inspect first, and what would convince you to add another cache layer?
Pick one answer.
What you gain, what you pay
- +50x latency reduction — 50 ms DB query becomes 1 ms cache hit.
- +Explicit per-object TTLs — fine-grained control over freshness.
- +Reduces database load 10-100x — the cache protects the DB.
- +Supports rich patterns (sorted sets, locks, rate limiters) when using Redis.
- +Decouples read latency from write cost — the DB can be optimized for writes, the cache for reads.
- −Cache-as-SPOF — when the cache fails, the database takes 10-100x its normal load.
- −Cache coherence complexity — invalidation bugs cause stale data.
- −Operational burden — running a distributed in-memory store is non-trivial.
- −Memory cost — RAM is 100x more expensive than disk per GB.
- −Application code is aware of the cache — abstractions leak (cache keys, TTLs, invalidation logic).
How this breaks in production
- Cache-as-SPOF — cache failure cascades into database overload.
- Stale data — invalidation fails or is forgotten, TTL too long.
- Cache stampede — popular key expiry causes thundering herd.
- Hot key — a single key receives disproportionate traffic, saturating one Redis shard.
- Big key — a single cache entry holds megabytes, causing latency spikes on read/eviction.
- Memory exhaustion — cache grows unbounded without eviction policy, OOM crashes.
Don't fall into these traps
- •No TTL on cached values — a single failed invalidation means stale data forever.
- •Updating cache before DB — if the cache SET succeeds but the DB write fails, you are inconsistent.
- •Forgetting cache stampede protection on hot keys — single-flight or probabilistic early expiration.
- •Sizing Redis/Memcached too small — eviction rate skyrockets, hit rate collapses.
- •Using the cache as a primary store — Redis is not a database; persistence is best-effort, not transactional.
- •Caching everything — caching rarely-read data wastes memory and adds invalidation complexity.
Real systems using this
How real systems implement this
- Twitter / X fanout-on-write timelines — Twitter pre-computes each user's timeline on write (fanout) and stores it as a Redis list. Reads are O(1) — just LRANGE on the user's list. This shifts the work from read time to write time and makes read latency uniform.
- GitHub's session and rendered-fragment caches — GitHub uses Redis extensively for session state, rate-limit counters, and rendered-fragment caches (Russian doll caching in Rails). The cache absorbs the bulk of read traffic; the database handles writes and cache misses.
- Discord's presence and channel state — Discord uses Redis for real-time presence, channel state, and message routing. The sorted set and pub/sub features of Redis are essential to the real-time architecture.
- Stack Overflow's multi-layer cache — Stack Overflow runs Redis in front of SQL Server, caching query results and rendered pages. They have publicly documented their cache strategy and hit rates — a textbook example of application caching at scale.
Practice saying it out loud
- Q1Design a caching layer for an e-commerce product page. What do you cache, with what TTLs, and how do you invalidate?
- Q2Your Redis cache crashes and the database collapses. What happened, and how do you prevent it next time?
- Q3When would you choose Redis over Memcached? When would Memcached be better?
- Q4A cached value is showing stale data intermittently. Walk through the possible causes.
- Q5Your cache hit rate is 60%. What does this tell you, and how do you improve it?
Further reading & references
Core explanations are original NO CAP material. External references are provided for deeper study and standards.
What next?
Mark as understood once the mental model clicks.
Next recommended
Database Caching