What is System Design
System design is the process of defining the architecture, components, modules, interfaces, and data flows of a software system to meet specific requirements for scale, reliability, performance, and maintainability. It is the difference between code that works on your laptop and systems that survive production.
None — start here.
How it works
System design answers a deceptively simple question: how do you build software that works at scale, survives failures, and can be maintained over time?
The question is simple because the goal is obvious. The answer is hard because 'scale', 'failure', and 'time' each pull in different directions.
When you write a script that reads a file and prints the result, you are programming. When you ask 'what happens if 10,000 people run this script at the same time, on different machines, and one of the machines catches fire?' — you are doing system design.
System design is the process of choosing what components your system has, how they communicate, where data lives, and what trade-offs you accept — so the system meets its requirements for scale, reliability, latency, cost, and maintainability.
When you design a system, you are making decisions in five areas:
1. Components — what services, databases, caches, queues, and storage systems exist. A simple blog might have one web server and one database. A system like Netflix has hundreds of microservices, multiple database clusters, CDNs across the globe, and message queues connecting everything.
2. Communication — how components talk to each other:
- Synchronous (HTTP, gRPC): the caller waits for a response. Simple, but couples the caller's latency to the callee's.
- Asynchronous (message queues, pub/sub): the caller sends and moves on. Decouples latency, but makes the system eventually consistent.
- Streaming (WebSockets, SSE): persistent connection, server pushes data. Real-time, but resource-intensive.
3. Data — where state lives and how it's stored:
- Relational (PostgreSQL): structured, ACID, JOINs. Good for transactions.
- Key-value (Redis): fast, simple. Good for caching.
- Document (MongoDB): flexible schema. Good for content.
- Wide-column (Cassandra): massive writes. Good for time-series.
- Object storage (S3): files, images, videos. Cheap, durable.
4. Failure — what happens when each component breaks:
- If the primary database dies, can a replica take over?
- If the cache cluster dies, does the system slow down or crash?
- If a downstream service is slow, does your service hang or fail fast?
- If an entire region goes offline, can you serve from another?
5. Scale — how the system behaves at different loads:
- 100 users: one server is fine.
- 10,000 users: you need load balancing, caching, and probably read replicas.
- 10,000,000 users: you need sharding, CDN, multi-region deployment, and careful capacity planning.
Every design decision is a trade-off. There are no free lunches.
You cannot have maximum consistency, maximum availability, and maximum performance simultaneously. You cannot have a system that is infinitely scalable, perfectly reliable, and trivially simple. System design is the art of choosing which trade-offs are acceptable for your specific problem.
Good system design always starts with the problem, not the technology. Before choosing Kafka or Cassandra, before drawing boxes and arrows, you must answer:
What problem am I solving?
- 'Users need to share short messages with followers.' (Twitter)
- 'Users need to watch videos on any device, anywhere.' (Netflix)
- 'Drivers and riders need to match in real time.' (Uber)
What are the constraints?
- How many users? (10K vs 10M changes everything)
- What latency is acceptable? (50ms vs 500ms vs 5s)
- What availability is required? (99% vs 99.99%)
- What's the budget? (startup vs enterprise)
- What's the team size? (2 people vs 200)
What are the trade-offs?
- Strong consistency vs high availability?
- Low latency vs low cost?
- Simplicity vs flexibility?
Only after answering these questions do you start choosing technologies. The biggest mistake in system design is jumping to 'use Kafka and Cassandra' before understanding whether you even need them.
In a system-design interview, the interviewer is not looking for a 'correct' answer — there usually isn't one. They are looking for:
- Clarification: do you ask good questions before designing? ('What's the DAU? What's the read/write ratio? Do we need real-time?')
- Structure: do you break the problem into requirements → capacity → API → data model → high-level design → deep dive → bottlenecks → trade-offs?
- Trade-off awareness: do you name what you're giving up when you choose a technology? ('I chose Redis for caching — it gives me sub-millisecond reads, but it's in-memory so I risk data loss on crash.')
- Communication: can you explain your reasoning out loud, draw diagrams, and adjust when the interviewer changes constraints?
This is why system design is a skill, not memorization. You practice it by designing systems, explaining your choices, and learning from real architectures.
Let's make this concrete. Suppose you're asked: Design a URL shortener like bit.ly.
- How many URLs per day? (100M/month → ~40 URLs/sec average, 400/sec peak)
- Read/write ratio? (100:1 — shorteners are read-heavy)
- Custom aliases? (Yes, but max 15 chars)
- Analytics? (Yes — track clicks, referrer, geo)
- Lifespan? (URLs never expire)
- 100M URLs/month × 100 bytes each = 10GB/month = 120GB/year of URL data
- 100:1 read ratio → 10B reads/month → ~4,000 reads/sec average
- Analytics: 10B click events/month → ~4,000 events/sec
POST /shorten {url: '...', alias?: '...'}→{short: 'https://x.co/abc123'}GET /{code}→ 301 redirect to long URL + record analytics
- URLs table:
code(PK),long_url,created_at,user_id - Clicks table:
code,timestamp,ip,referrer,geo - Store: SQL for URLs (need uniqueness + transactions), analytics DB (Cassandra — high write throughput)
- Clients → CDN → Load Balancer → App servers → (Cache for hot URLs + DB for cold)
- Analytics: App → Message Queue → Analytics consumer → Cassandra
- How to generate short codes? Hash + base62 encoding, or counter + encoding.
- How to handle collisions? Check DB, retry with salt.
- How to serve 4,000 reads/sec? Cache hot URLs in Redis (hit rate > 90%).
- How to handle analytics without slowing redirects? Fire-and-forget to a queue.
This is system design: structured thinking about trade-offs at scale.
Which statement best captures what system design is?
Pick one answer.
You're asked to 'Design Twitter.' What should you do FIRST?
Pick one answer.
In a system design interview, what is the interviewer PRIMARILY evaluating?
Pick one answer.
Real architectures evolve with scale — Netflix as a worked example.
Netflix started in 1997 as a single monolithic application talking to a single Oracle database — the kind of system one team could deploy on a Friday afternoon. That worked while Netflix was a DVD-by-mail service shipping a few thousand discs a day. Then they launched streaming in 2007, and the geometry of the problem changed overnight: a DVD shipment is one event per user per week; a streaming session is hundreds of requests per user per hour, from anywhere on earth, on any device, with sub-second latency expectations.
The evolution (documented across a decade of Netflix engineering blog posts and postmortems):
- 2008–2010: monolith → horizontally-scaled stateless web tier behind AWS Elastic Load Balancer; Oracle replaced with sharded MySQL for the billing system that needed ACID; Cassandra introduced for viewing-history (massive writes, eventual consistency tolerated).
- 2010–2013: full migration to AWS — their own datacenters shut down. Per-service microservices replaced the monolith. EVCache (a Memcached-derived distributed cache) absorbs 90%+ of reads. Zuul (reverse proxy / API gateway) sits in front of all services. Hystrix (circuit breaker) isolates per-service failures.
- 2013–2018: global expansion drives multi-region active-active. Cassandra clusters span AWS regions. Chaos Monkey randomly kills production instances to force engineers to design for failure — the origin of the Simian Army chaos-engineering practice.
- 2018–present: Spinnaker (continuous delivery) does thousands of deployments per day. Each service owns its own data store; no shared database across services. Edge traffic flows through Amazon's global CDN plus their own Open Connect appliances inside ISP facilities.
The Netflix story is not 'we used microservices and Kafka'. It is 'each step of growth forced a specific architectural change, and each change was a trade-off.' Going from monolith to microservices bought independent deploys and per-service scaling at the cost of distributed-systems complexity, eventual consistency between services, and a much harder debugging story. EVCache bought a 90% cache hit rate at the cost of an in-memory store that, when it crashes, drops a multi-gigabyte-per-second read load straight onto Cassandra. Every architecture decision at Netflix is a trade-off, and the trade-offs compound.
Scaling changes the system at every order of magnitude.
A useful mental model: each 10x in load forces a different architectural decision. Missing the transition kills companies.
- 100 → 1,000 users (a single server works): you are programming, not designing. Focus on correctness and ship.
- 1,000 → 10,000 users (single server strains): introduce a load balancer + a second app server, add a cache for hot reads, offload static assets to a CDN. The database is still one box.
- 10,000 → 100,000 users (database becomes the bottleneck): add read replicas, move sessions to Redis, move file uploads to S3. App servers are now stateless.
- 100,000 → 1,000,000 users (single-region ceiling): shard the database, add message queues for async work, introduce a service mesh. Deploy across multiple AZs.
- 1,000,000 → 10,000,000 users (one region is not enough): go multi-region. Active-active with conflict resolution, geo-routed traffic, global CDN. Each region now contains a full stack.
- 10,000,000+ (the global internet-scale tier): custom protocols (QUIC), edge compute (Cloudflare Workers, Lambda@Edge), purpose-built databases (Spanner, DynamoDB), chaos engineering as a routine practice.
The common failure mode is jumping two steps ahead — adopting Kafka and sharded Cassandra at 5,000 users because 'we'll need it eventually'. You won't, or you will need something different by the time you do. The second-most-common failure mode is refusing to take the next step — running a single MySQL box at 500,000 users because 'it's simpler'. Both are expensive, in opposite directions.
Every senior design conversation ends with an explicit trade-off matrix. For each option, name (a) what it gives you, (b) what it costs, and (c) under what condition it breaks. Example: 'Redis cache gives sub-millisecond reads, costs in-memory RAM ($/GB) and risks data loss on crash, breaks when the working set exceeds RAM or when invalidation falls behind writes.' If you can't fill in column (c), you don't yet understand the choice. The most common design failure is not picking the wrong technology — it's picking a technology without knowing when it will betray you.
In a system design interview, after you sketch a CDN → load balancer → stateless app services → Cassandra diagram for 'Design Netflix', the interviewer asks: 'So you'd just use Cassandra everywhere?' What is the strongest response?
Pick one answer.
Engineering mental model
Mental model. Think of What is System Design as a deliberate boundary in a system. The boundary exists because something becomes harder to manage when everything is done in one place: latency, scale, failure isolation, consistency, cost, or team ownership. The useful question is not “what does What is System Design mean?” but “what pressure makes this boundary worth introducing, and what new failure mode does it create?”
Before choosing What is System Design, name the workload, the critical user path, the dominant bottleneck, the failure you are trying to absorb, and the trade-off you are willing to accept. If you cannot name those five things, the design is probably premature.
// Pseudocode
request = receive()
result = what_is_system_design(request)
return result
// Production questions:
// 1. What happens on timeout?
// 2. Can this operation be retried safely?
// 3. What is the bottleneck?Back-of-the-envelope reasoning
Numerical lens: write down traffic, payload size, read/write ratio, peak multiplier and durability target before choosing a component. The numbers should justify the architecture.
Interactive thought experiment: What is System Design
Change the variables below and predict what breaks first in What is System Design. The production lab can later reuse these same inputs.
Change one variable at a time. Predict the failure mode first, then move the slider and see whether your mental model matches the simplified system response.
If you are stuck on What is System Design, start by drawing the request path and marking every network hop, stateful component, queue, cache and failure boundary. Then estimate where the system will saturate.
You increase traffic by 10× in a system using What is System Design. What should you inspect first?
Pick one answer.
Which statement is the safest engineering habit when using What is System Design?
Pick one answer.
You have dashboards for traffic, latency, errors and saturation. You can change the architecture, but every change has operational cost.
Production scenario: your system uses What is System Design, traffic suddenly spikes, and p99 latency doubles. What is your first move?
Interview drill
Answer this without notes: When would you choose What is System Design, and when would you intentionally avoid it? Mention at least one bottleneck it addresses, one failure mode it introduces, and one alternative. Then quantify the workload you are designing for.
A useful engineering lens for What is System Design: define the problem it solves, the simpler design that fails first, the constraint that forces you to introduce this concept, and the new failure modes the concept creates.
Numerical sanity check
Back-of-the-envelope reasoning beats fake precision. State your traffic, payload, concurrency and growth assumptions explicitly, then calculate enough to know whether the current architecture is orders of magnitude away from the target.
Do not optimize for a memorized definition. Reason from the workload and failure mode.
Imagine the simplest version of a system using What is System Design. What breaks first as traffic grows by 10×, and what would you change before reaching 100×?
Pick one answer.
What you gain, what you pay
- +Forces explicit reasoning about scale, reliability, and cost before building.
- +Catches problems before they become production incidents.
- +Creates a shared language for engineering teams to discuss architecture.
- +Enables teams to make informed decisions about technology choices.
- −No single 'correct' answer — judgement-based and subjective.
- −Time-consuming; easy to over-engineer or under-engineer.
- −Requires broad knowledge of many technologies and patterns.
- −Hard to practice without real production experience.
How this breaks in production
- Over-engineering: adding queues, caches, and microservices before the system needs them. Premature complexity is worse than simple code.
- Under-engineering: assuming a single server and single database will scale forever. They won't.
- Cargo-culting: copying a big-company architecture (e.g., Netflix) without understanding why they made those choices. Netflix has different constraints than you.
- Ignoring failure modes: designing only for the happy path. In production, the unhappy path is the normal path.
Don't fall into these traps
- •Jumping into a solution before clarifying requirements and constraints.
- •Memorizing architectures instead of understanding trade-offs. 'Use Kafka' is not a design — it's a technology choice that must be justified.
- •Treating 'scalability' as a single axis — it has throughput, latency, and data-volume dimensions. A system can be high-throughput but high-latency.
- •Forgetting that the simplest design that meets requirements is usually the best design. Complexity has a cost.
Real systems using this
How real systems implement this
- Netflix — Microservices architecture with hundreds of services, each designed for a specific scale and failure profile. They migrated from a monolith to microservices to enable independent scaling and deployment. Their architecture is documented openly in their tech blog.
- Uber — Domain-oriented microservices with explicit dispatch, geospatial, and pricing subsystems. Evolved from a monolith to a distributed system over years as they scaled from one city to hundreds. Their engineering blog details the evolution.
Practice saying it out loud
- Q1What is system design, and why does it matter?
- Q2Walk me through how you would approach designing a system you've never seen before.
- Q3What's the difference between system design and software architecture?
- Q4Why do we study system design separately from programming?
Further reading & references
Core explanations are original NO CAP material. External references are provided for deeper study and standards.
What next?
Mark as understood once the mental model clicks.
Next recommended
How to Approach System Design