Design Chat System

What are we designing? A general-purpose real-time chat system like Slack or Discord. Users join workspaces (servers), belong to channels (rooms), and exchange messages in real time. The system supports typing indicators, presence ('online now'), message edits/deletes, threaded replies, file attachments, and search. Unlike WhatsApp (mobile-first, push-driven), chat systems are often desktop-first with always-open apps that hold the connection for hours.

The defining challenge is presence and typing at scale: a single Slack workspace can have 100K users across 10K channels, and every keystroke in a channel must be broadcast to every other viewer of that channel within 200 ms. Multiply by every active channel, every active user, and you have a brutal fan-out problem on a tiny, latency-sensitive payload.

Functional requirements.

• Send a message to a channel (text + optional files + optional thread).
• Receive messages in real time when the channel is open.
• Typing indicators ('Alice is typing...').
• Presence: see who's online in the workspace.
• Edit / delete messages.
• Threaded replies.
• Search across message history.
• Push notifications when offline or mentioned.

Non-functional requirements.

• Message delivery latency: p99 < 200 ms for online channel members.
• Typing broadcast latency: < 200 ms.
• Presence update latency: < 5 s.
• Availability: 99.95% per workspace (a workspace outage is a business stopper).
• Durability: no message loss; history preserved for years.
• Scale: 10M+ concurrent users across 1M+ workspaces; some workspaces have 100K+ members.

Non-goals. No voice/video calls (separate system), no workflow automation (Slack Workflows, etc.).

Capacity estimation.

Messages. Assume 50M messages/day globally. Each message ~1KB (text + metadata + file refs). 50M x 1KB = 50 GB/day = ~18 TB/year. Tiny by modern standards.

Concurrent connections. 10M concurrent users, each holding 1 WebSocket for their session. A chat server holds ~100K connections (Slack uses ~50K/host). Need ~100-200 chat servers globally.

Typing indicators. Each user in an active channel types ~10 chars/sec while composing, generating ~1 typing event/sec. With 1M active channels averaging 5 viewers each = 5M typing broadcasts/sec. This dwarfs message traffic — typing is the real load.

Presence. 10M users, presence updates every 30 s = 333K presence events/sec. Each event broadcasts to every workspace member viewing the member list — say 100 viewers per workspace = 33M presence broadcasts/sec at the extreme. We need to throttle / batch presence updates.

Search. 50M messages/day indexed. Search index ~5x message size = 250 GB/day. Elasticsearch cluster sharded by workspace.

Storage. Messages stored in sharded SQL (per-workspace Postgres) or Cassandra. 5-year retention: 50M x 365 x 5 = 91B messages x 1KB = ~91 TB. Fits in Cassandra.

APIs. The chat system uses WebSocket for real-time frames and REST for non-real-time operations.

WS  /v1/connect  (auth token, workspace_id in handshake)
  -> frame: MSG_SEND    { channel_id, text, thread_id?, client_msg_id }
  <- frame: MSG_ACK     { server_msg_id, ts }
  <- frame: MSG_RECV    { channel_id, sender_id, text, server_msg_id, ts }
  -> frame: TYPING      { channel_id, is_typing }
  <- frame: PRESENCE    { user_id, status }
  -> frame: MSG_EDIT    { server_msg_id, new_text }
  -> frame: MSG_DELETE   { server_msg_id }

POST /v1/messages          (REST fallback for offline send)
GET  /v1/channels/:id/messages?before=...
GET  /v1/search?q=...&channel=...
POST /v1/files             (presigned S3 URL for upload)

The WebSocket is the primary transport; REST is used when no connection is open (mobile notifications reply via REST) and for paginated history fetches.

Data model.

Messages (Cassandra, partitioned by (workspace_id, channel_id), clustered by server_msg_id):

messages (
  workspace_id  BIGINT,
  channel_id    BIGINT,
  server_msg_id TIMEUUID,    -- Snowflake, sortable
  sender_id     BIGINT,
  text          TEXT,
  thread_id     BIGINT NULL,
  edited_at     TIMESTAMP NULL,
  deleted_at    TIMESTAMP NULL,
  file_urls     LIST<TEXT>,
  ts            TIMESTAMP,
  PRIMARY KEY ((workspace_id, channel_id), server_msg_id)
)

Channels (per-workspace SQL):

channels (id, workspace_id, name, type ENUM('public','private','dm'), created_at)
channel_members (channel_id, user_id, joined_at)

Connection registry (Redis, mapping user_id -> set of chat_server_ids):

key: conn:{user_id}
value: SET of chat_server_ids (a user may have multiple devices)
TTL: 60s

Channel subscribers (Redis, mapping channel_id -> set of online user_ids):

key: subs:{channel_id}
value: SET of user_ids currently viewing the channel
TTL: 300s   -- refreshed by client heartbeats

When a message arrives for channel C, we lookup subs:C and broadcast to each subscriber's chat server.

Presence (Redis):

key: presence:{workspace_id}
value: HASH { user_id: status (active|away|offline) }

Updates throttled to every 30s per user.

Deep dive: presence, typing, and the broadcast fan-out.

The hardest part of a chat system is not storing messages — that's a solved problem (Cassandra, partitioned by channel). The hard part is broadcasting small, high-frequency events (typing, presence, edits) to every viewer of every channel, within 200 ms, without melting the system.

The subs:C set. When a user opens channel C, their client subscribes via the chat server, which does SADD subs:{C} user_id. When they switch channels, SREM from the old, SADD to the new. The chat server reads subs:C to know who to broadcast to. TTL 300s means a closed laptop stops receiving broadcasts within 5 minutes.

Typing throttling. A naive implementation sends a typing event on every keystroke. At 10 chars/sec per user, this is 10 broadcasts/sec per typing user per channel. We throttle to 1 event/sec per (user, channel) — clients render an 'Alice is typing...' indicator that fades after 3 seconds, so 1Hz is enough to keep it alive.

Presence batching. Presence updates for 10M users at 30s interval = 333K events/sec. Each event broadcasts to every workspace member viewing the member list — potentially 100K viewers for a large workspace. Naive: 33M broadcasts/sec. We batch: presence updates are aggregated per workspace and pushed every 5s; clients render a slightly-stale member list.

Multi-device fan-out. A user with 3 devices (desktop, phone, tablet) has 3 WebSocket connections, possibly to 3 different chat servers. The conn:{user_id} key holds a SET of chat_server_ids. Every broadcast to user X must reach all 3 servers. We do this by sending to each chat_server_id in the set.

Message edits and deletes. Edits broadcast an MSG_EDIT frame to subs:C with the new text and server_msg_id; clients replace the message locally. Deletes broadcast MSG_DELETE; clients replace the message with '(deleted)'. The Cassandra row is updated with edited_at / deleted_at; reads filter deleted messages.

Search indexing. Every message publish also goes to a Kafka topic per workspace; an indexer writes to Elasticsearch. Search returns message_ids; the client fetches full messages from Cassandra. Search is decoupled from the chat path — a search outage doesn't break chatting.

Idempotency and ordering. Same as WhatsApp: client_msg_id for idempotency; Snowflake TIMEUUID for ordering within a channel. Channel messages are clustered by server_msg_id, so reads are in send order.

Bottlenecks and failure modes.

• Typing broadcast storm. A 1000-person channel where everyone is typing generates 1000 broadcasts/sec to 1000 viewers = 1M fan-out writes/sec for one channel. Mitigation: cap typing broadcasts to the first 50 viewers (you can't render 1000 typing indicators anyway); sample typing events at 1Hz per user.

• Chat server failure. All connections on a dead chat server must reconnect. Mitigation: sticky-session rebalancing; clients reconnect with exponential backoff; new chat server re-subscribes to all the user's open channels.

• Hot channel. A 100K-person company-wide channel where every message broadcasts to 100K subscribers. Mitigation: broadcast via a tree (the sender's chat server forwards to N 'aggregator' chat servers, each of which fans out to their local subscribers).

• Redis failure. Without Redis, we can't look up subscribers; broadcasts fail. Mitigation: Redis cluster with replicas; fall back to broadcasting only to users on the sender's chat server (degraded mode).

• Search index lag. Recently posted messages aren't searchable for 5-30s. Mitigation: accept it; show 'this message is being indexed' for the sender.

• Push notification spam. A user mentioned in 5 channels while offline generates 5 separate push notifications. Mitigation: coalesce notifications into one 'you have 5 new mentions' push, with deep links.

• Workspace-level isolation failure. A bug in one workspace affecting another. Mitigation: shard by workspace_id; per-workspace rate limits; per-workspace keyspaces in Cassandra.

• Message edit race. Two devices edit the same message simultaneously. Mitigation: last-write-wins by edited_at timestamp; clients fetch the final version on reconnect.

Scaling strategy and trade-offs.

Chat server scaling. Add chat servers horizontally. Use sticky routing (hash of user_id -> chat server) so reconnects are cheap. A dead chat server's users reconnect with backoff.

Redis cluster. Shard by user_id for conn: keys, by channel_id for subs: keys, by workspace_id for presence. Cross-shard operations (e.g. broadcasting to a channel whose subscribers span many shards) require fan-out at the application layer.

Cassandra scaling. Shard by (workspace_id, channel_id). Each channel's messages live on one partition (cheap reads). For very active channels (company-wide), sub-partition by day.

Multi-region. Run chat servers in 5+ regions. Each user connects to nearest. Cross-region broadcasts go through a backbone RPC. Messages are written to a single-region Cassandra primary and asynchronously replicated.

Search scaling. Elasticsearch cluster per workspace (or per-shard for large workspaces). Use rollover indices (one per month) so old indices can be searched lazily.

Trade-offs made explicit.

• We chose per-connection stateful chat servers — gained low-latency broadcasts, lost rebalancing ease (mitigated by sticky routing + client backoff).
• We chose Cassandra for messages — gained write throughput and time-series friendliness, lost JOINs (hydrating a thread requires a separate query).
• We chose 1Hz typing throttle — gained bounded broadcast load, lost UI responsiveness (acceptable; clients render the indicator for 3s).
• We chose 5s presence batch — gained presence traffic manageable, lost real-time presence accuracy (acceptable; users won't notice 5s staleness).
• We chose async search indexing — gained no impact on chat latency, lost instant search for very recent messages.

Mental model. Think of Design Chat System as a deliberate boundary in a system. The boundary exists because something becomes harder to manage when everything is done in one place: latency, scale, failure isolation, consistency, cost, or team ownership. The useful question is not “what does Design Chat System mean?” but “what pressure makes this boundary worth introducing, and what new failure mode does it create?”

Numerical lens: write down traffic, payload size, read/write ratio, peak multiplier and durability target before choosing a component. The numbers should justify the architecture.

Answer this without notes: When would you choose Design Chat System, and when would you intentionally avoid it? Mention at least one bottleneck it addresses, one failure mode it introduces, and one alternative. Then quantify the workload you are designing for.

Numerical sanity check

Back-of-the-envelope reasoning beats fake precision. State your traffic, payload, concurrency and growth assumptions explicitly, then calculate enough to know whether the current architecture is orders of magnitude away from the target.