Web Server Caching

Web server caching (also called reverse proxy caching) places a cache inside the reverse proxy that fronts the application tier. When a request arrives, the proxy checks its local cache; on a hit, it serves the cached response directly, never touching the application. On a miss, it forwards to the application, caches the response, and serves it.

This sits between the CDN (which is geographically distributed, at edge POPs) and the application cache (Redis, in-process). It is the layer that absorbs cacheable traffic that slipped past the CDN — typically because the user is in a region with no nearby POP, or because the CDN evicted the entry, or because the request was not CDN-eligible (e.g., it carried an authorization header).

client → CDN edge → reverse proxy cache (NGINX/Varnish) → application tier → Redis → DB

The classic tools are NGINX (with the proxy_cache directive), Varnish (purpose-built for HTTP caching, configured via VCL — Varnish Configuration Language), and Apache Traffic Server. Cloud variants include AWS CloudFront edge caches (which double as both CDN and web-cache) and managed offerings like Cloudflare Cache Reserve.

The two big differences from a CDN: (a) the web server cache is in your data center, so latency to it is the same as to your app tier (single-digit ms), and (b) you have full programmatic control — VCL lets you write logic that no CDN would let you deploy (e.g., strip cookies, rewrite URLs, vary cache key by user segment).

VCL — the Varnish superpower.

Varnish Configuration Language (VCL) is what sets Varnish apart. VCL is a compiled DSL that lets you write arbitrary logic for every request and response — inspect headers, rewrite URLs, change cache keys, bypass the cache conditionally, purge by pattern, and more. NGINX has proxy_cache directives with similar (though less expressive) capabilities.

Typical VCL patterns:

• Strip cookies from static asset requests so that Cookie: session=abc does not bust the cache for /static/app.js.
• Vary cache key by user segment — cache a different version for logged-in vs logged-out users, or for users in different countries.
• Bypass cache for POST/PUT/DELETE — only cache idempotent requests.
• Grace mode — serve stale content when the origin is slow or down (stale-while-revalidate baked in).
• Saint mode — temporarily blacklist an unhealthy origin, preventing cache misses from reaching it.
• PURGE — invalidate individual URLs by request from the application tier.

The key insight is that the web server cache is the most programmable cache layer. The CDN is at the edge but config-light (you mostly set headers and hope). The app cache is per-object but in your application code. The web server cache is in the data path and fully programmable, which is why Varnish is the favorite of high-traffic sites that need fine-grained caching logic.

Cache keys — what counts as the same request.

The cache key determines whether two requests hit the same cached response. The default is (method, host, URL) — but this is rarely sufficient.

Common cache key considerations:

• Query parameters. ?utm_source=email should not bust the cache for a static page. VCL/NGINX config typically normalizes query params, stripping tracking params before computing the key.
• Cookies. Logged-in users see different content, so cookies must be part of the key OR the response must not be cached. Static asset requests should have cookies stripped.
• Headers. Accept-Encoding: gzip vs no-encoding produce different responses — the cache key must include encoding. So must Accept-Language if the response is localized.
• Authorization. A request with Authorization: Bearer ... should NOT be cached by default (it is user-specific). Either bypass the cache for authed requests, or vary the cache key by user.

The most common web-cache bug: caching a logged-in user's response under a shared key, then serving it to a different user. The fix is either (a) strip cookies and only cache truly public responses, or (b) include the user ID in the cache key. Most production VCLs do both: cache aggressively for cookieless requests, bypass for authed requests.

When to use web server caching:

• Any website serving public, cacheable HTTP responses. Default-on for most deployments.
• Sites with a heavy application tier that you want to shield from cacheable traffic.
• APIs with idempotent GET endpoints that return mostly-public data.
• When you need fine-grained caching logic (per-segment cache keys, conditional purges) that the CDN does not allow.
• When you need a graceful failure mode — grace mode serves stale content while the app recovers.

When NOT to use web server caching:

• Personalized responses (per-user dashboards, real-time data) — the cache hit rate will be near zero, and you risk cross-user leakage.
• POST/PUT/DELETE responses — non-idempotent, must reach the app.
• Endpoints that require authentication and return user-specific data. Bypass the cache or use per-user keys.
• Low-traffic internal services where the app tier can handle all traffic directly. The proxy adds operational complexity.

Mental model. Think of Web Server Caching as a deliberate boundary in a system. The boundary exists because something becomes harder to manage when everything is done in one place: latency, scale, failure isolation, consistency, cost, or team ownership. The useful question is not “what does Web Server Caching mean?” but “what pressure makes this boundary worth introducing, and what new failure mode does it create?”

Numerical lens: write down traffic, payload size, read/write ratio, peak multiplier and durability target before choosing a component. The numbers should justify the architecture.

Answer this without notes: When would you choose Web Server Caching, and when would you intentionally avoid it? Mention at least one bottleneck it addresses, one failure mode it introduces, and one alternative. Then quantify the workload you are designing for.

Numerical sanity check

A useful first-order model is cache_load = request_rate × (1 - hit_rate). If traffic is 20,000 req/s and the hit rate is 90%, roughly 2,000 req/s still reaches the origin before considering misses caused by expiration or eviction.