Build the mental models behind real systems. 144 seed concepts across 16 areas.
Alerts and visualization are the user-facing layer of observability: dashboards make the system's state visible at a glance, and alerts notify humans when something needs attention. Good alerting is hard — alert on too much and on-call engineers burn out from fatigue (alert deafness); alert on too little and outages go undetected. The rule: every alert must be actionable, investigated, and either fixable or silenced.
An Anti-Corruption Layer (ACL) is a translation boundary between two bounded contexts — typically a new system and a legacy one — that prevents the legacy's domain model, terminology, and quirks from leaking into the new system. The ACL translates in both directions: outbound calls from the new system become legacy-compatible calls, and inbound responses from the legacy become clean new-domain objects. The new system never sees the legacy's data shapes, error conventions, or technical debt — it sees only its own clean model. The ACL is the price you pay to keep a migration from corrupting the system you're migrating to.
Good API design is the difference between a system developers love and one they tolerate. The principles: RESTful resource modeling, clear versioning, sensible pagination, consistent error handling, and idempotency for safety. A good API is consistent (same conventions everywhere), predictable (you can guess endpoints), and forgiving (idempotent so retries are safe). A bad API is the opposite — surprising, inconsistent, and dangerous to retry.
An API Gateway is a single entry point that sits in front of all your backend services. It routes requests to the right service, terminates TLS, enforces authentication and rate limiting, transforms requests and responses, and shields clients from the internal topology. Think of it as the front desk of your microservices hotel: every visitor checks in here, gets vetted, and is escorted to the right room — without ever learning the building's floor plan.
Application caching stores computed results, objects, or rendered fragments in an in-memory store (Redis, Memcached) accessed by application code. It is the most common cache layer — the one engineers think of when they say "add a cache." Application caching is explicit, per-object, and flexible: the application decides what to cache, how to compute the key, and what TTL to set. The cost is cache coherence complexity, the cache-as-single-point-of-failure risk, and the operational burden of running a separate distributed store.
The application layer is the part of your system that executes business logic: it receives a request, validates input, orchestrates calls to databases and downstream services, applies the rules that make your product meaningful, and returns a response. In a well-designed service, application servers are stateless — they hold logic and orchestration, while state lives in dedicated stores (databases, caches, object storage).
Async request-reply decouples a slow operation from the client that initiated it. The client sends a request, immediately receives an identifier (a correlation ID or job ID), and later retrieves the result by polling a status endpoint or receiving a callback. The request never holds a connection open while the work happens; instead, the work proceeds on a queue and the result is materialized separately. This pattern is how every long-running API works — report generation, video transcode, model training, batch export.
Authentication (AuthN) verifies who you are — distinguishing a user from an attacker. Methods range from passwords (knowledge-based) to OAuth tokens and biometrics. The hard parts aren't the verification itself but the lifecycle around it: password storage, session management, multi-factor authentication, and token revocation. Get authentication wrong and attackers can impersonate any user; get it right and even a database leak is survivable.
Authorization (AuthZ) decides what an authenticated user is allowed to do. Two dominant models: RBAC (role-based — "users with the admin role can delete") and ABAC (attribute-based — "users in the finance dept can view invoices over $10k from their region"). AuthN establishes identity; AuthZ uses it. Every request must be authorized, not just authenticated, or authenticated users can act as each other.
Availability monitoring measures whether users can actually reach and use your service — from outside the data center, not just from inside. It uses synthetic probes (Periodic HTTP requests from global locations) to detect outages that internal health checks miss: DNS failures, load balancer misconfigurations, region routing issues, TLS expiry. The output is uptime (e.g., 99.95% over 30 days) and the alerting is paged when availability drops below the SLO.
Availability patterns are the architectural moves you make so that the failure of any single component does not take down the system. The core ideas are redundancy (multiple copies of everything), replication (kept in sync), failover (automatic switch when one copy dies), and graceful degradation (serving partial results when dependencies degrade). High availability is not a property you add at the end — it is a property you design in from the first box.
Availability means every request gets a response. Consistency means every read sees the latest write. In distributed systems, you often have to choose between them — especially during network partitions. This trade-off is the heart of CAP theorem.
Back pressure is when a downstream component signals upstream to slow down because it can't keep up. Without it, a slow consumer causes unbounded queue growth, OOM crashes, and cascading failures. With it, the system degrades gracefully under load.
A background job is a unit of work performed outside the request/response cycle. Instead of doing slow, expensive, or non-critical work synchronously while the user waits, the application enqueues a job (with a message broker or database row) and returns immediately. Workers pick up the job asynchronously and execute it. Patterns include fire-and-forget, scheduled jobs (cron), deferred work, and retries on failure.
Bandwidth estimation computes how much network egress the system produces per second: QPS × payload size, converted from bytes/sec to Gbps (1 Gbps = 10^9 bits/sec = 125 MB/sec). The estimate determines whether the system needs a CDN, multiple regions, or peer-to-peer distribution. Bandwidth is often the dominant constraint in media-heavy systems — a single origin server cannot physically serve 100 Gbps, and even 10 Gbps is rare. Forgetting to convert bytes to bits (8x error) is the most common bandwidth-estimation mistake.
A bottleneck is the single slowest stage in a request path — the resource whose utilization hits 100% first and caps throughput for the whole system. Identifying it requires measuring every stage (profiling), driving real load (load testing), and watching the system in production (APM). The bottleneck governs capacity: until you fix it, every other optimization is wasted. After you fix it, the bottleneck moves — and you measure again.
A bulkhead isolates resources so a failure in one part of the system doesn't take down everything else. Like watertight compartments in a ship's hull, if one compartment floods, the ship stays afloat. In software: separate thread pools, connection pools, or processes for different services or workloads.
Cache aside is the most common caching strategy. The application checks the cache first; on a miss, it fetches from the database, writes to the cache with a TTL, and returns. Writes update the DB and invalidate the cache. It is simple, fault-tolerant, and wastes no memory on unread data — but it allows stale reads and cache stampedes.
Cache sizing determines how much RAM to dedicate to the cache layer. The key insight is the Pareto distribution (80/20 rule): a small fraction of keys generates the majority of traffic, so caching just the hot set gives a high hit rate with modest memory. The standard methodology: estimate the working set (hot 1-5% of total data), size the cache to fit it (plus headroom), and verify the hit rate empirically. Over-sizing wastes RAM; under-sizing causes cache thrashing. Cache sizing is the fourth of the capacity-estimation skills and the trickiest because it depends on access distribution, not just totals.
Caching stores frequently accessed data in faster storage. The strategy you choose — cache-aside, write-through, write-behind, or refresh-ahead — determines when data is written to the cache, how stale it can be, and what happens during failures. Choosing the wrong strategy causes stale data, cache stampedes, or data loss.
CAP is the most quoted — and most misunderstood — theorem in distributed systems. A distributed system can provide at most two of three guarantees: Consistency, Availability, Partition tolerance.
Capacity estimation is the back-of-the-envelope math that turns "design Twitter" into concrete numbers: 200M DAU × 50 reads/day = 10B reads/day ≈ 115k QPS average, 5-10x peak. You estimate QPS, storage, bandwidth, and cache size from user counts and behavior, then size each system component accordingly. The math is approximate (order-of-magnitude is the goal), but it is the single most distinguishing skill in system design interviews — it separates hand-waving from engineering.
CDN caching stores responses at edge nodes geographically close to users, so a request for static (or stale-while-revalidate-able) content is served from the nearest edge instead of the origin. Latency drops from 100+ ms (cross-continent) to 5-30 ms (local edge), origin load collapses to a small fraction of read traffic, and the system becomes resilient to origin failures. The cost is cache coherence: every edge holds its own copy, and invalidation must propagate to hundreds of POPs.
A Content Delivery Network (CDN) is a globally distributed set of cache servers (POPs) that store copies of your static and cacheable content close to users. Instead of every user fetching each asset from a single origin server across the world, they fetch from a nearby edge node, cutting latency from 200ms to under 20ms and shielding the origin from traffic spikes.
A circuit breaker stops calling a failing service after a threshold of failures, giving it time to recover. It has three states: closed (normal), open (failing, stop calling), and half-open (testing if recovery happened). It prevents cascading failures and retry storms.
The Claim Check pattern stores a large message payload in external storage (object store, database) and passes only a small reference (the claim check) through the message queue. The consumer uses the reference to retrieve the full payload when needed. This keeps messages small (queues are optimized for small, fast messages), avoids message size limits, and reduces broker storage costs. The pattern is named after the claim check you get at a coat check — a small token representing a large object stored elsewhere.
Client caching stores responses on the client — browser, mobile app, or desktop — so that subsequent reads are served locally without touching the network at all. It is the cheapest, fastest, and most distributed cache layer: zero server load, zero network latency, infinite horizontal scale (every client is its own cache). The cost is invalidation: once a response is on a client, the server cannot easily retract or update it, so client caches must be designed around TTLs, versioned URLs, and explicit cache-busting strategies.
A compensating transaction undoes the effects of a previously committed transaction by executing a reverse operation — not by rolling back. It's the rollback mechanism of the Saga pattern: when a step in a multi-service workflow fails, you call compensations for each prior step in reverse order. Unlike a database rollback, compensations are business-level operations that work across services and don't require distributed locks.
Competing Consumers is the parallelism pattern for a work queue: multiple consumer instances pull from the same queue, and the broker ensures each message is handed to exactly one of them. Throughput scales horizontally with the number of consumers up to the broker's partition limit. The catches are ordering (you lose it across consumers), idempotency (a redelivered message may hit a different consumer), and backpressure (more consumers do not help if the bottleneck is downstream).
Consensus is the problem of getting multiple distributed nodes to agree on a single value despite crashes, message loss, and reordering. Paxos (Lamport, 1998) and Raft (Ongaro & Ousterhout, 2014) are the two algorithms that solve this safely: they guarantee that the agreed value is never wrong (safety) and that the system eventually agrees as long as a majority of nodes can communicate (liveness, modulo FLP). Every strongly consistent distributed database, lock service, and configuration store is built on one of these or a variant.
Consistency patterns describe the guarantees a distributed system makes about what data readers observe after writes. The three families — weak, eventual, and strong — sit on a spectrum trading latency, availability, and complexity against correctness. Understanding which pattern your system needs is the difference between a chat app that feels snappy and a bank ledger that loses money.
Consistent hashing is a distributed hashing technique that minimizes data movement when nodes are added or removed. With naive modulo hashing (hash(key) % N), removing one node re-maps almost every key. With consistent hashing, only the keys on the removed node need to move. This is how Cassandra, DynamoDB, and Redis Cluster distribute data.
CQRS (Command Query Responsibility Segregation) separates the model used to *write* data from the model used to *read* data. Instead of one shared schema optimized for neither, you have a write model optimized for transactional integrity and a read model optimized for queries — kept in sync by events. The two models can live in different stores (e.g., Postgres for writes, Elasticsearch for reads), be scaled independently, and use different shapes (normalized vs denormalized). The trade-off is operational complexity: you now have two systems to keep consistent instead of one.
Data modeling is the art of representing real-world entities and relationships in a database. The key decisions: entities (what tables), relationships (one-to-one, one-to-many, many-to-many), normalization vs denormalization (consistency vs read speed), primary keys (natural vs surrogate), and indexes (what to index, what not to). Good data modeling makes queries fast and invariants enforceable; bad modeling makes everything slow and inconsistent.
Database caching is the cache layer built into the database engine itself — PostgreSQL's `shared_buffers` and OS page cache, MySQL's InnoDB buffer pool, the (deprecated) MySQL query cache, SQL Server's plan cache. It is the deepest cache layer, transparent to the application, and the foundation that makes modern relational databases usable. The trade-off is that you have very little control: the engine chooses what to cache based on access patterns, and tuning is limited to sizing the buffer and choosing storage layouts.
Denormalization trades storage and write complexity for read performance by duplicating data so that reads can be answered from a single place. It is the standard pattern for scaling read-heavy workloads, NoSQL data modeling, and precomputing query results. The cost is update amplification — when the source data changes, every copy must be updated — and the resulting risk of inconsistency.
The Deployment Stamps pattern deploys a complete, independent copy of a system — application, storage, configuration — as a self-contained unit called a stamp (or ‘cell’, ‘pod’). Each stamp serves a slice of users or workload; multiple stamps run in parallel for scale-out, isolation, or geographic distribution. A stamp is the unit of deployment, scaling, failure, and recovery. By partitioning users across stamps, you bound the blast radius of any failure to one stamp and scale horizontally by adding stamps.
Design a Slack/Discord-style real-time chat system at 10M concurrent users. Covers stateful chat servers with sticky WebSocket routing, the subs:{channel_id} Redis set for channel subscriber tracking, throttled typing broadcasts (1Hz, capped to first 50 viewers), batched presence updates (5s), multi-device fan-out via conn:{user_id} SET, Cassandra for messages sharded by (workspace_id, channel_id), and Elasticsearch for search fed async by Kafka.
Design a Dropbox-style file storage and sync system. Covers chunked uploads with resumable transfer, content-addressed storage with SHA-256 chunk deduplication, a metadata DB mapping files to chunk hashes, async block-fetch sync to other devices, and operational-transform / last-writer-wins conflict resolution. The deep dive walks through why dedup at the chunk level saves 30-70% of storage in real corpora and how the 'selective sync' client avoids downloading everything to every device.
Design a hotel booking system (Booking.com / Expedia). Covers inventory as a (hotel_id, room_type_id, date) cube, the concurrency problem of two users booking the last room (pessimistic vs optimistic locking), overbooking as a deliberate strategy, the search-and-filter query for available hotels by date range, and the reservation state machine with payment + cancellation flows. The deep dive walks through why naive `SELECT FOR UPDATE` on per-date rows is too coarse and how to model inventory at the date grain with version counters.
Design a photo-sharing social network with 500M DAU. Covers presigned-S3 photo upload with async transcoding, the feed-generation decision between fan-out-on-write vs fan-out-on-read (and the hybrid used for celebrities), Redis sorted-set feed caches, and CDN-fronted media delivery. The deep dive walks through why pure fan-out-on-write collapses for Justin Bieber and how the hybrid model solves it.
Design a Dynamo-style distributed, eventually-consistent key-value store. Covers consistent hashing with virtual nodes, replication factor N with tunable W/R quorums, vector clocks for concurrent-write detection, hinted handoff for partition tolerance, read repair, and Merkle-tree anti-entropy. The deep dive walks through how a put/get traverses the ring and how the system stays available during node failures.
Design Netflix at 250M subscribers, 30% of US internet traffic. Covers the transcoding pipeline (master mezzanine -> 15 encodings per title -> HLS/DASH segments -> DRM), the Open Connect CDN (appliances inside ISPs), adaptive bitrate streaming (player switches bitrate per segment based on throughput), and pre-computed recommendations refreshed nightly. The deep dive walks through why Netflix built its own CDN and how ABR works.
Design a ranked (Facebook-style) news feed at 2B users. Covers pre-computing ranked feeds every 5 min so reads stay O(1) despite expensive ML ranking, the candidate-vs-ranked feed split in Redis, the feature store for ranking signals, the hide-feedback loop that adapts ranking per user, and the celebrity-bypass hybrid fan-out for the candidate set.
Design a multi-channel notification system (iOS/Android push, SMS, email, in-app) that fans a single event out to many channels and many recipients. Covers the event bus -> fan-out worker pattern, per-provider rate limiting (APNS/FCM, Twilio, SES), template rendering with localization, idempotency keys to defeat retries, preference & quiet-hours enforcement, and the priority queue that keeps 'your ride is here' ahead of '20% off shoes'.
Design a distributed rate-limiting service that enforces per-user, per-IP, per-route limits at 1M req/sec with sub-millisecond added latency. Covers token bucket vs sliding window vs leaky bucket, a Redis-backed global counter store with Lua scripts for atomicity, an L1 local cache to short-circuit abuse, and fail-open vs fail-closed trade-offs per route.
Design Uber / Lyft's ride-matching system. Covers geospatial indexing with geohash and quadtree, the nearest-driver query (k-NN on a moving set of points), the rider->driver broadcast-and-accept protocol, supply-demand balancing via surge pricing, and trip state machine. The deep dive walks through why a naive `SELECT * FROM drivers ORDER BY distance` collapses at scale and why quadtree + Redis geo-index is the standard answer.
Design a full-text search system (Elasticsearch / Lucene architecture). Covers the inverted index data structure, analyzer pipeline (tokenize -> lowercase -> stem -> stop words), term-sharded indexing, TF-IDF + BM25 ranking, the query pipeline (parse -> plan -> scatter-gather across shards -> merge), and near-real-time indexing via the commit/refresh/flush cycle. The deep dive walks through why you shard by term (not by document) for skewed term frequencies, and how per-shard postings lists are merged with a priority queue at query time.
Design Twitter (now X) at 250M DAU, 500M tweets/day. Covers the celebrity fan-out problem (Elon's tweet to 150M followers), hybrid fan-out-on-write with celebrity bypass, Snowflake IDs for time-sortable globally-unique tweet IDs, Cassandra for tweet storage, Redis ZSET timeline caches, and Elasticsearch search fed asynchronously by Kafka.
Design Uber at 15M trips/day, 5M drivers, 1M+ concurrent trips. Covers Redis GEO sorted sets for nearest-driver queries, H3 hex cells for surge zones, 1-3s driver location updates, parallel dispatch to top-3 drivers with atomic Lua-script concurrency control, surge pricing recomputed every 5 min from demand/supply streams, GPS map-matching with Kalman filtering, and per-city geospatial sharding.
Design a bit.ly-style URL shortener that turns long URLs into short codes, serves millions of redirects per second with single-digit-millisecond latency, and tracks click analytics. The case study walks through hash-based short codes, 301 vs 302 redirects, a Redis read-through cache for hot URLs, and the trade-offs between base-62 encoding, counter-based IDs, and collision handling.
Design a video streaming system (Netflix / YouTube). Covers the HLS / DASH adaptive bitrate protocols, the transcoding pipeline (source upload -> encode multiple resolutions -> manifest), CDN delivery with edge caching, and the client-side ABR controller that switches bitrates based on bandwidth. The deep dive walks through why we transcode to multiple resolutions (adaptive bitrate over flaky mobile), why segments are 10-second chunks (vs 1-second or 1-minute), and how the manifest file (HLS .m3u8 / DASH .mpd) ties everything together.
Design WhatsApp at 2B users, 100B messages/day. Covers long-lived WebSocket connection fabric (1M conns per chat server in Erlang/Go), presence-in-Redis routing, message ordering via Snowflake TIMEUUIDs, idempotent sends via client_msg_id, single/double/blue tick delivery receipts, group fan-out optimization (batch presence + batched RPCs per chat server), 30-day server retention, and E2E encryption (server never sees plaintext).
Design YouTube at 2B users, 500 hours uploaded/minute. Covers resumable multipart upload to GCS, a chunked parallel transcoding pipeline producing 10 encodings per video (5 resolutions x 2 codecs), HLS/DASH adaptive bitrate playback via multi-tier CDN, search via Elasticsearch fed async by Kafka, and recommendation ranking via a candidate-then-rank two-stage model with pre-computed top-N cached in Redis. The deep dive walks through how transcoding keeps up with 500 hours/minute.
Disaster recovery (DR) is the plan for when an entire site, region, or service is lost — not a single failed instance. It's defined by two metrics: RTO (recovery time objective — how long until we're back up) and RPO (recovery point objective — how much data we can lose). DR strategies range from cold standby (cheap, slow) to hot multi-region (expensive, fast). The right choice depends on the cost of downtime vs the cost of preparedness.
A distributed lock provides mutual exclusion across multiple processes running on different machines, so only one of them can perform a critical section at a time. The naive implementation — set a key in Redis with a TTL — is unsafe under GC pauses, network partitions, and clock skew. Safe distributed locking requires either a consensus-based lock service (etcd, ZooKeeper, Chubby) or the Redlock algorithm with fencing tokens to defend against stale-lock holders. The general lesson is that distributed locking is harder than it looks and should usually be avoided in favor of idempotency or single-leader designs.
A distributed system is one in which multiple independent computers cooperate over a network to appear as a single coherent service to the outside world. The defining property is not geographical spread but partial failure: components you depend on can fail independently and unpredictably, and the network that connects them can drop, delay, or reorder messages at any time. Mastering distributed systems is largely about learning to design as if every assumption you would make on a single machine — shared memory, synchronous calls, reliable clocks — does not hold.
A distributed transaction is a unit of work that spans multiple independent resources — different databases, message brokers, or services — and must either commit on all of them or roll back on all of them. The classical solution is Two-Phase Commit (2PC): a coordinator asks every participant to prepare, then to commit, blocking until both phases complete. The modern solution for microservices is the saga pattern: a sequence of local transactions, each with a compensating action that undoes its effect if a later step fails. Both have steep costs — 2PC sacrifices availability and latency for strong consistency; sagas sacrifice isolation for resilience.
DNS is the phonebook of the internet. Humans remember example.com; routers need 93.184.216.34. DNS bridges this gap with a hierarchical, distributed, eventually-consistent database.
Document stores (MongoDB, CouchDB, Couchbase, Elasticsearch) persist self-describing, JSON-like documents keyed by ID. Each document can have a different shape — fields can be added or removed without a schema migration. They excel when your data is naturally hierarchical, your read pattern is 'give me the whole aggregate by ID', and your write pattern is 'replace this document'. They struggle when you need cross-document JOINs or strict referential integrity.
Event-driven architecture (EDA) is a pattern where services communicate by producing and consuming events, rather than calling each other directly. Producers emit events ('user created', 'order placed') without knowing who consumes them. This enables loose coupling, easy extensibility, and independent scaling.
Event Sourcing stores the system's state as an append-only log of events — every state change is captured as an immutable event, and current state is derived by replaying the log. Instead of storing the current row in `orders`, you store `OrderCreated`, `OrderShipped`, `OrderCancelled` events; the order's current state is computed by folding over the events. This gives an audit trail for free, time-travel queries (what was the state at 3 PM Tuesday?), and the ability to rebuild any read model from the log. The trade-offs: complexity, eventual consistency on reads, and an event schema that must be carefully versioned.
An External Config Store moves configuration out of the application code or deployment artifact into a separate service the application reads at runtime. This enables changing behavior — feature flags, thresholds, endpoint URLs, kill switches — without rebuilding or redeploying. The config store becomes the control plane for the fleet: one update propagates to thousands of instances in seconds, with audit history, gradual rollouts, and instant rollback. Without it, every behavioral change requires a deployment, which is slow, risky, and impossible during incidents.
Failover is the process of switching to a redundant standby when the primary fails — promoting a replica database, redirecting traffic to a healthy region, or routing around a failed instance. The hard part isn't the switch itself; it's deciding when to fail over, doing it without split-brain, and recovering when the original primary comes back. Automatic failover trades speed for risk of false positives; manual failover trades speed for safety.
Failure analysis is the discipline of asking "what will break?" before something does. It identifies single points of failure (SPOFs), cascading failure paths, and bottlenecks under load. Techniques: component-by-component review ("what if this dies?"), dependency mapping, capacity analysis ("what if traffic doubles?"), and game days (deliberate failures in production). The output isn't just a list of risks — it's a prioritized plan for what to harden first.
Federated identity lets a user authenticate with one identity provider and use that identity to access multiple services — without each service managing passwords. "Sign in with Google" and enterprise SSO (Okta, Azure AD) are federated identity. Standards: SAML (enterprise), OIDC (modern web), OAuth (delegated access). The benefit: one strong identity, many relying services; password fatigue and reuse disappear.
Federation (also called functional partitioning or vertical partitioning by function) splits a database by domain — users in one database, posts in another, messages in a third — so each can scale, fail, and evolve independently. Unlike sharding (which splits one table's rows across machines), federation splits by function: every database holds a different slice of the schema, and the application stitches them together.
The gatekeeper pattern places a single, hardened gateway between clients and backend services. The gateway validates requests (authentication, authorization, schema, rate limits), rejects invalid ones, and forwards only clean traffic to the backends. Backends trust the gateway and don't repeat the checks. This concentrates security in one well-audited place rather than spreading it (inconsistently) across every service.
The Geodes pattern deploys backends in multiple geographically distributed regions, all active simultaneously, with users routed to the nearest one. Unlike active-passive multi-region (where one region handles traffic and the other stands by), Geodes is active-active: every region serves users, every region has a full copy of the data, and a region failure just shifts its users to another region. The pattern gives low latency to global users, survives region outages, and scales linearly with the number of regions — at the cost of data replication complexity.
Graceful degradation means: when a dependency fails, serve partial functionality instead of crashing. A page that loads with stale recommendations is better than a page that doesn't load at all. The pattern requires identifying which features are essential vs optional, having fallbacks for each, and making sure the fallback path is actually exercised.
Graph databases (Neo4j, Amazon Neptune, Dgraph, TigerGraph, ArangoDB) store data as nodes and edges — first-class relationships. Where SQL needs expensive recursive JOINs to traverse connections, graph databases traverse relationships in O(1) per hop using index-free adjacency. They excel at social networks, recommendation engines, fraud detection, and any domain where 'how are these things connected?' is the dominant question.
GraphQL is a query language for APIs where the client describes the exact shape of the data it wants, and the server returns exactly that — no more, no less. It solves the two chronic REST problems — over-fetching (you GET a giant user object when you needed just the name) and under-fetching (you fetch a user, then their posts, then their comments — three round trips). GraphQL trades server-side complexity for client-side flexibility and performance.
gRPC is a modern RPC framework built by Google on HTTP/2 and Protocol Buffers. It is typed, binary, streaming-capable, deadline-aware, and codegen-driven for a dozen languages. It is the de-facto standard for internal service-to-service communication at scale because it is dramatically faster and more expressive than JSON-over-HTTP, while still being a real protocol with real tooling.
Health monitoring continuously checks whether a service is functioning, typically via a /health endpoint that returns 200 OK or an error. The two probes — liveness ("is the process alive?") and readiness ("is the process ready to serve traffic?") — drive orchestration decisions: restarting a dead container vs removing a not-ready one from the load balancer. Confusing them is one of the most common causes of cascading failures in Kubernetes.
Horizontal scaling means adding more machines (nodes) to handle more load, rather than making one machine bigger. It is the primary way modern systems scale — but it requires statelessness, distributed data, and careful failure handling.
The internet is a network of networks. Your device does not connect directly to a server — it goes through routers, ISPs, DNS resolvers, CDNs, and finally the origin. Understanding this chain is the foundation for every system-design decision.
A structured approach to system design: clarify requirements → estimate capacity → define APIs → model data → design high-level architecture → deep-dive components → identify bottlenecks → discuss trade-offs. Following this skeleton prevents you from jumping straight into a solution and missing critical constraints.
HTTP is the application-layer protocol that powers the web. It is request-response, stateless, and text-based (in HTTP/1.1). Understanding HTTP methods, status codes, headers, and the evolution from HTTP/1.1 to HTTP/2 to HTTP/3 is fundamental to every web system design.
An operation is idempotent if doing it once has the same effect as doing it many times. In distributed systems where retries and duplicates are normal, idempotency is not optional — it's the difference between safe retries and double charges.
The Index Table pattern creates secondary lookup tables for NoSQL stores that don't natively support secondary indexes. The primary store is keyed by one access pattern (e.g., user_id), but queries often need to look up by other attributes (e.g., email, username, last_login_date). The index table is a separate, smaller table that maps the secondary attribute back to the primary key, maintained on every write. The result: O(1) lookup-by-email on a NoSQL store that would otherwise require a full scan.
Instrumentation is the code that emits metrics, logs, and traces from inside your application — the difference between "the service is slow" and "the db.insert call in /orders is taking 800ms." Modern instrumentation is built on OpenTelemetry, structured logging, and consistent labeling. Without instrumentation, observability tools have nothing to show; with good instrumentation, every question about production is answerable in minutes.
Isolation levels control how concurrent transactions see each other's effects. The ANSI SQL standard defines four levels — Read Uncommitted, Read Committed, Repeatable Read, and Serializable — each preventing specific anomalies (dirty reads, non-repeatable reads, phantom reads, serialization anomalies). Higher isolation means more correctness but less concurrency. Most databases default to Read Committed, which is weaker than people assume.
Key-value stores are the simplest NoSQL model: a flat dictionary mapping keys to opaque values. Lookups are O(1) hash operations, writes are append-friendly, and the engine makes almost no assumptions about what the value contains. Redis and DynamoDB are the canonical examples, but they sit on opposite ends of the consistency/ durability spectrum — Redis is an in-memory cache, DynamoDB is a durable, replicated, multi-region store.
Latency is how long one operation takes. Throughput is how many operations happen per unit time. They are related but distinct: you can have high throughput with high latency (batch processing) or low latency with low throughput (single-user app). Knowing which one to optimize is a core system design skill.
Leader election is the process by which a set of distributed nodes choose exactly one of themselves to act as the coordinator — the leader that accepts writes, drives replication, or makes scheduling decisions. Having a single leader avoids the cost of full consensus on every operation: once elected, the leader can act unilaterally until it fails. The hard parts are detecting failure reliably, preventing split-brain (two leaders), and ensuring the new leader sees all committed data before serving traffic. Algorithms include Bully, Ring, and the term-based voting used in Raft.
A load balancer distributes incoming traffic across multiple servers. It is the foundational scaling primitive: it enables horizontal scaling, fault tolerance, and rolling deploys. Without it, you have one server and a single point of failure.
Logical clocks order events in a distributed system without relying on wall-clock time. Lamport timestamps assign a monotonically increasing counter to every event such that causally-related events are correctly ordered. Vector clocks extend this to detect concurrent events — two events that did not causally influence each other. Both are necessary because clock skew between machines makes wall-clock timestamps unreliable for ordering, and because some operations (CRDT merges, conflict detection in Dynamo, dependency graphs in Spanner) must reason about causality, not about 'when' something happened.
A materialized view is a pre-computed, stored result of a query, refreshed periodically or on change. Unlike a regular SQL view (which is just a saved query re-executed on read), a materialized view holds actual data — so reads are fast (a single table scan or key lookup instead of a complex join) but writes bear the cost of keeping the view up to date. The pattern trades write cost and staleness for read speed and simplicity, and is the workhorse pattern behind every dashboard, every search index, and every read-optimized projection in a CQRS system.
A message queue decouples a producer from a consumer by inserting a durable buffer in between. Producers write messages and immediately return; consumers pull and process at their own pace. This buys you traffic smoothing, producer-consumer independence, retry semantics, and a dead-letter escape hatch for poison messages — at the cost of latency, infrastructure, and the obligation to make every consumer idempotent.
A message queue decouples producers from consumers. Producers write messages to a queue; consumers read them at their own pace. This smooths traffic spikes, enables async processing, and lets producers and consumers scale independently.
Metrics, logs, and traces are the three pillars of observability. Metrics are aggregated numeric signals (cheap, queryable, good for alerting). Logs are discrete events (rich context, expensive to store, good for debugging). Traces follow a single request across services (causality, latency breakdown). A mature observability stack uses all three — they answer different questions.
Microservices are an architectural style in which a system is built as a set of small, independently deployable services, each owning its own data store and communicating over a network (usually HTTP, gRPC, or async messages). The promise is independent team velocity; the cost is a dramatic increase in operational and integration complexity. Microservices are a trade-off, not a default — and they are usually the wrong choice for small teams and young products.
Multi-region architecture deploys a service across multiple geographic regions for lower latency to global users and survival of region-level outages. The two topologies are active-passive (one region serves traffic; the other stands by) and active-active (both serve live traffic). The hard parts aren't deploying to two regions — they're data replication, consistency models, and data residency compliance.
Multi-Version Concurrency Control (MVCC) is how PostgreSQL, MySQL/InnoDB, Oracle, and SQL Server (with RCSI) handle concurrent reads and writes without locking readers. Each transaction sees a consistent snapshot of committed data as of its start time; writers create new versions of rows instead of overwriting in place. Readers never block writers and writers never block readers — a major concurrency win over the lock-based alternatives.
OAuth 2.0 is the standard for delegated authorization: letting a third-party app access a user's data on another service without sharing the user's password. The user authenticates with the provider (Google, GitHub), the provider issues a token to the app, the app uses the token to access the API. Key flows: authorization code (web apps), client credentials (service-to-service), PKCE (mobile/SPA). Tokens, not passwords, are the access mechanism.
Object storage is a storage model optimized for files (objects) of any size, accessed by a flat key (the object name) over HTTP. It is cheap, durable (typically 11 nines — once written, your data is essentially never lost), infinitely scalable (no capacity planning), and accessed via a simple REST API. S3 is the canonical example; GCS, Azure Blob, R2, and MinIO all follow the same model. It is the default home for files, images, videos, backups, logs, and any data that does not need block-level random access.
Performance monitoring tracks how fast your system is: latency (how long requests take), throughput (how many you handle), and error rate (how many fail). The three golden signals (plus saturation) make up the USE/RED method. The key tool is the latency histogram and its percentiles — p50 (median), p95 (95th percentile), p99 — because averages hide the long tail that affects real users.
Performance is how fast a system handles a single request. Scalability is how well it handles more requests. They are related but distinct: a system can be fast and unscalable, or slow and scalable. Confusing them leads to bad architecture decisions.
Pipes & Filters decomposes a complex processing task into a sequence of small, single-purpose stages (filters) connected by channels (pipes). Each filter reads input, transforms it, and writes output — without knowing what's upstream or downstream. Unix pipes (`cat file | grep | sort | uniq`) are the canonical example; modern equivalents include ETL pipelines, stream processing (Kafka Streams, Flink), and CI/CD workflows. The pattern trades monolithic complexity for composability: filters can be reused, reordered, parallelized, and tested independently.
A priority queue processes messages by priority rather than arrival order. High-priority messages jump the queue ahead of low-priority ones, ensuring that critical workloads — premium users, payment failures, security alerts — get handled first even under sustained load. Implementation can be N separate FIFO queues polled in priority order, a single queue with a priority field and a heap-based consumer, or weighted fair queuing. The pattern trades FIFO simplicity for SLA-aware processing.
Pub/sub is a messaging pattern where producers (publishers) send messages to topics, and consumers (subscribers) receive messages from topics they subscribe to. Unlike a work queue (where each message goes to one consumer), pub/sub delivers each message to ALL subscribers.
QPS (queries per second) estimation converts daily active users and behavior into the request rate the system must handle: DAU × actions/user/day ÷ 86400 = average QPS. The same formula split by operation type gives read QPS vs write QPS, the most important architecture-driving ratio. Peak QPS is 3-5x average for normal workloads and 10-100x for event-driven spikes. QPS estimation is the first and most central of the four capacity-estimation skills.
Queue-Based Load Leveling inserts a queue between a task producer and a task consumer to smooth traffic spikes. The producer writes to the queue at whatever rate it receives work; the consumer reads at a steady rate it can sustain. The queue absorbs the difference — bursts fill the queue, lulls drain it — so the consumer never sees the burst and never gets overwhelmed. The result: predictable load on the consumer, no resource over-provisioning for peak, and graceful degradation under sustained overload (queue grows, latency rises, but the system doesn't crash).
QUIC is a UDP-based transport protocol that rebuilds TCP's reliability, congestion control, and TLS's encryption in userspace, while fixing TCP's biggest modern problem: head-of-line blocking across multiplexed streams. QUIC powers HTTP/3 and now carries a meaningful fraction of the internet's traffic. It is the most significant transport-layer innovation since TCP itself.
A quorum is the minimum number of nodes that must participate in an operation for it to be considered valid. The fundamental rule is R + W > N: the read quorum and write quorum must overlap, so any read sees the latest write. For a 5-node cluster, a quorum is 3 — the smallest majority that cannot simultaneously agree on two conflicting values. Quorums are the math behind every consensus protocol, every replicated write, and every dynamic-membership decision; understanding them is the difference between 'it works on my laptop' and 'it survives a partition.'
Rate limiting caps how many requests a client can make in a time window. It protects services from abuse (DDoS, scraping), ensures fair resource sharing, and prevents cascading failures. Common algorithms: token bucket, leaky bucket, fixed window, sliding window.
Real-time communication pushes data to clients as it happens, rather than waiting for them to ask. Four technologies dominate: WebSockets (bidirectional, persistent), Server-Sent Events (server-to-client only, simpler), WebRTC (peer-to-peer, audio/video), and long polling (HTTP fallback). Choosing the right one depends on directionality (one-way vs two-way), latency requirements, browser support, and infrastructure complexity.
Refresh-ahead proactively refreshes popular cache entries before they expire, so reads of hot keys always hit a fresh value and never trigger a thundering herd. The cache itself (or a background worker) tracks TTL countdowns and re-fetches from the database a configurable margin before expiry. The result: hot keys never miss, latency is uniformly low, and stampede risk collapses — at the cost of background DB load and wasted refreshes for keys that are no longer hot.
Replication copies data from a primary database to one or more replicas. It improves read availability, read throughput, and fault tolerance. The main trade-offs are replication lag (eventual consistency) and write amplification.
REST is an architectural style for APIs that uses HTTP verbs on resources identified by URLs. It is stateless, cacheable, and uniform. Most public web APIs use REST (or something close to it) because it is simple, widely understood, and works over standard HTTP.
Retrying failed requests is the simplest reliability pattern — but done wrong, it makes failures worse. Retries must be idempotent, bounded, and use exponential backoff with jitter to avoid retry storms.
A reverse proxy is a server that sits in front of one or more backend servers and forwards client requests to them. To the client it looks like the proxy is the server; to the backend it looks like the proxy is the client. Reverse proxies terminate TLS, load-balance, cache, rewrite URLs, throttle, and shield backends from direct exposure. NGINX, HAProxy, Envoy, and Caddy are the household names; most production web services have at least one.
RPC is the idea that calling a function on a remote machine should look and feel like calling a function in your own process. The RPC framework hides marshalling, the network, and the protocol behind a generated stub that mimics a local function. It is the natural fit for service-to-service communication where both ends are owned by the same team and the API is action-oriented rather than resource-oriented.
A saga is a sequence of local transactions, each on a different service, where every step has a compensating action that semantically undoes its effect if a later step fails. Unlike Two-Phase Commit, sagas never block — the system stays available under failure — but they sacrifice isolation: intermediate states are visible to other transactions, and the application must be designed to handle them. Sagas are the de facto standard for cross-service transactions in modern microservice architectures and the answer most senior engineers give to 'how do you do distributed transactions?'
The Sequential Convoy pattern processes a sequence of related messages in strict order per partition, while allowing parallelism across partitions. Messages with the same key (e.g., all events for one order, all transactions for one account) are routed to the same partition and processed serially — guaranteeing that `OrderCreated` is handled before `OrderShipped`. Messages with different keys go to different partitions and process in parallel. The pattern gives you the simplicity of ordered processing within a logical group without sacrificing fleet-wide throughput.
Server-Sent Events is a standard for one-way streaming from server to client over plain HTTP. The server holds the response open and writes `text/event-stream` chunks; the browser's `EventSource` API parses them and fires JavaScript events. It is dramatically simpler than WebSockets when the server only needs to push — no upgrade, no binary framing, automatic reconnection with replay built into the protocol.
Service discovery is the mechanism by which services locate each other in a dynamic environment where instances come and go. It replaces hard-coded IP addresses and DNS with a registry that tracks live instances and answers 'where is the auth service right now?'. Two flavors: DNS-based (simple, slower to propagate) and registry-based (Consul, etcd, ZooKeeper, Kubernetes API — fast, watchable, often paired with client-side or server-side load balancing).
A service mesh is an infrastructure layer that handles service-to-service communication, built from two planes: a data plane (sidecar proxies like Envoy intercepting all traffic) and a control plane (a central API that configures the proxies). The mesh provides mTLS, retries, circuit breaking, traffic shifting, observability, and policy uniformly across a polyglot fleet — without applications knowing it exists. Service code stops worrying about operational concerns; the mesh handles them. Istio and Linkerd are the canonical open-source implementations; AWS App Mesh and Consul Connect are alternatives.
Sharding splits a database into smaller pieces (shards) distributed across multiple machines. Each shard holds a subset of the data. This enables horizontal scaling of both reads and writes — but adds complexity in routing, cross-shard queries, and rebalancing.
A sidecar is a helper container deployed alongside the main application container, sharing the same pod (or host) and lifecycle but running as a separate process. It handles cross-cutting concerns — networking, observability, configuration, security — so the application code can focus on business logic. The pattern decouples operational infrastructure from the application: change the proxy or logger without recompiling the app; reuse the same sidecar across services written in different languages. Envoy, the canonical sidecar, is the data plane of every major service mesh.
A Single Point of Failure (SPOF) is any component whose failure takes down the whole system. Identifying SPOFs means walking every request path and asking 'if this one thing dies, does the system die with it?' Elimination is achieved through redundancy (N+1, active-active), graceful degradation, and failover design — but every redundancy has a cost, so you eliminate the SPOFs that matter and consciously accept the rest.
SLI (Service Level Indicator) is a measurement: "99.5% of requests succeeded." SLO (Service Level Objective) is the target: "we aim for 99.9% success over 30 days." SLA (Service Level Agreement) is the contract: "if we miss 99.5%, we refund you." The error budget — the gap between the SLO and 100% — is the controlled resource that lets teams balance reliability against feature velocity.
SQL tuning is the discipline of making slow queries fast. The toolkit is small and durable: indexes (B-tree, partial, composite), the query planner, EXPLAIN ANALYZE, connection pooling, and eliminating N+1 queries. Mastering it means thinking in sets, understanding what the planner sees, and never trusting intuition without measurement.
SQL databases (PostgreSQL, MySQL) store data in tables with strict schemas and support ACID transactions. NoSQL databases (Cassandra, MongoDB, DynamoDB) trade strict consistency for scalability, flexibility, or performance. The choice depends on your data shape, consistency requirements, and scale.
Storage estimation computes how much disk the system will consume over a retention period. The base formula: writes/day × payload size × retention × replication factor = total storage. Text and structured data are small (KB-MB per record); media dominates (MB-GB per object). The estimate determines whether the data fits in a database, requires object storage, or requires sharding — and how much it will cost. Storage estimation is the second of the four capacity-estimation skills.
Strangler Fig is a migration pattern: instead of rewriting a legacy system in one risky big-bang, you wrap it with a routing layer that incrementally redirects new features (or specific endpoints) to a new implementation, while the old system keeps running. Over time, the new system 'strangles' the old one — taking on more responsibility — until the legacy can be retired entirely. The pattern, named by Martin Fowler after a strangler fig vine that grows around and eventually replaces its host tree, trades a high-risk big-bang for a series of low-risk, reversible steps.
A task queue decouples producing work from doing work. Producers push tasks onto a queue; workers pull them off and execute. This enables background jobs (email sending, image processing), smooths bursty load (the queue absorbs spikes), and lets you scale workers independently of web servers. Tools: Celery (Python), Sidekiq (Ruby), SQS (managed), Redis-backed queues.
TCP is the connection-oriented, reliable, ordered transport protocol that carries most of the internet's traffic — HTTP, HTTPS, gRPC, SSH, SMTP, databases, message queues. It achieves reliability on top of an unreliable IP network through sequence numbers, acknowledgements, retransmission, flow control, and congestion control. Understanding TCP is understanding why your HTTP request takes 100ms before it can even send a byte.
Throttling is a server-side mechanism that limits how fast a client can send requests, smoothing bursts and protecting downstream resources. It uses the same algorithms as rate limiting (token bucket, leaky bucket) but is enforced at the consumer or service boundary, often dynamically based on system health. The key distinction: rate limiting rejects excess traffic with 429s; throttling delays or shapes it.
A timeout is the maximum time you're willing to wait for an operation. Every external call — HTTP request, database query, RPC — must have one, or a slow dependency becomes a system-wide outage. The key distinctions are connection timeout vs read timeout, client-side vs server-side timeout, and ensuring deadlines propagate across the call chain.
TLS is the protocol that encrypts data in transit and authenticates the parties on each end. It is what makes HTTPS work, what protects your database connection, what secures your email, and what every 'green padlock' in your browser represents. Without TLS, every byte you send across the internet can be read and modified by anyone in the path. With TLS, those bytes are encrypted, integrity-protected, and (optionally) authenticated on both sides.
A transaction is a sequence of database operations treated as a single, indivisible unit of work. ACID — Atomicity, Consistency, Isolation, Durability — is the set of guarantees that make transactions safe. Single-database transactions are well understood; the moment a transaction spans multiple databases or services, ACID breaks down and you must adopt alternatives like sagas or two-phase commit.
Two-Phase Commit (2PC) is the classical protocol for atomic commitment across multiple resources. A coordinator asks every participant to PREPARE in phase one; if all vote YES, the coordinator sends COMMIT in phase two. The protocol guarantees atomicity (all commit or all abort) but is blocking — if the coordinator crashes between phases, participants hold locks indefinitely waiting for a decision they cannot make on their own. Three-phase commit (3PC) removes the blocking but adds a round trip and requires bounded network delays. In practice, microservices have largely abandoned 2PC in favor of the saga pattern.
UDP is the connectionless, unreliable, fastest transport in the IP family. You hand it a datagram, it sends it, and that is the entire contract. No handshake, no retransmission, no ordering, no flow control. This makes UDP the right choice when late data is worse than missing data — voice, video, gaming, DNS — and the foundation on which modern transports like QUIC and WebRTC are built.
The Valet Key pattern gives clients temporary, limited-scope credentials to access a storage system (typically object storage like S3) directly, bypassing the application server. Instead of streaming a 5 GB upload through your service, the service mints a presigned URL that lets the client write directly to S3 for the next 15 minutes, scoped to one bucket key. The pattern is named after hotel valet keys — limited-functionality keys that open doors and start the ignition but not the trunk or glovebox. The result: lower service bandwidth, lower latency, and the service stays stateless.
Vertical scaling (scale up) means making a single machine bigger — more CPU, more RAM, faster disks — instead of adding more machines. It is the simplest scaling strategy: no code changes, no distributed systems, no statelessness required. The trade-off is a hard ceiling (the biggest available instance), downtime to upgrade, and a single point of failure. Vertical scaling is the right first step for most systems; horizontal scaling becomes necessary when you hit the vertical ceiling or need fault tolerance.
The Write-Ahead Log (WAL) is the durability mechanism at the heart of every transactional database. Before any change is applied to the data files, a record of that change is appended to the WAL and fsynced. If the database crashes, the WAL is replayed on restart to bring the data files back to a consistent state — committed transactions survive, uncommitted ones don't. The WAL also powers streaming replication, point-in-time recovery, and transactional outboxes.
Web server caching stores HTTP responses in a reverse proxy that sits in front of the application tier. NGINX, Varnish, and Apache Traffic Server intercept requests, check a local cache, and serve cached responses without involving the application at all. This drops application load dramatically, slashes latency for cacheable requests, and gives the application a graceful failure mode (serve stale when the app is down). It is the layer between the CDN and the application, governing how much read traffic ever reaches the app tier.
WebRTC is a free, open framework for real-time voice, video, and data communication between browsers, mobile apps, and IoT devices — often directly peer-to-peer, without routing media through a server. It bundles a media engine (codecs, echo cancellation, jitter buffers), a secure transport (DTLS over SRTP), and a NAT-traversal toolkit (ICE, STUN, TURN) into one browser-native API. It is the technology behind Google Meet, Zoom's web client, Discord voice, and every in-browser video call.
WebSockets is a protocol that upgrades a single HTTP connection into a persistent, full-duplex, bidirectional TCP-like channel between client and server. Once upgraded, either side can send messages to the other at any time without opening new HTTP requests. It is the standard solution for chat, multiplayer games, collaborative editing, live dashboards — any app where the server needs to push data to the client with sub-second latency.
System design is the process of defining the architecture, components, modules, interfaces, and data flows of a software system to meet specific requirements for scale, reliability, performance, and maintainability. It is the difference between code that works on your laptop and systems that survive production.
Wide-column stores (Cassandra, HBase, ScyllaDB, Bigtable) store data in sparse, sorted, multi-dimensional maps: rows are keyed by a partition key, and within a partition, columns are sorted by a clustering key. They are optimized for massive write throughput (millions of writes/sec across a cluster) and time-series access patterns, at the cost of JOINs, ad-hoc queries, and strong cross-row consistency.
Write-behind (also called write-back) makes every write go to the cache only and returns to the caller immediately. The cache then propagates the write to the database asynchronously, often batched and throttled. Writes are blindingly fast — the caller never waits for the DB — but the cache is now the source of truth for a brief window, and if the cache crashes before flushing, recent writes are lost. Write-behind is the highest-throughput, highest-risk caching strategy.
Write-through caching makes every write go to the cache and the database synchronously, in the same request. The cache is therefore always fresh: there is never a stale-read window. The cost is higher write latency — every write now pays the round-trip to both stores — and a partial-failure window in which one write succeeds and the other fails. Write-through is the right choice when reads must never be stale and the write path can tolerate the extra milliseconds.