WebSockets — Full-Duplex Real-Time Over TCP
WebSockets is a protocol that upgrades a single HTTP connection into a persistent, full-duplex, bidirectional TCP-like channel between client and server. Once upgraded, either side can send messages to the other at any time without opening new HTTP requests. It is the standard solution for chat, multiplayer games, collaborative editing, live dashboards — any app where the server needs to push data to the client with sub-second latency.
How it works
HTTP is half-duplex and request-driven: the client asks, the server answers. The server cannot push anything unless the client asks first. For most apps this is fine. For real-time apps — chat, gaming, live dashboards — it is fatal. You cannot render a stock price 200ms late; you cannot show a chat message only when the user happens to refresh.
Three pre-WebSocket workarounds existed, all bad:
- Polling: client sends
GET /messagesevery 1s. Wastes bandwidth and battery; latency floor of the poll interval. - Long polling: client sends
GET /messagesand the server holds the connection open until a message arrives, then responds. Better, but every message costs a new HTTP request with headers, and the server must track pending requests. - HTTP streaming (chunked transfer): the server keeps the response open and writes chunks. Works but fights proxies and has awkward framing.
WebSockets fixes all three. The connection starts as an HTTP request with an Upgrade: websocket header; the server responds with 101 Switching Protocols; from then on, the TCP connection is a full-duplex, framed, bidirectional channel. Both sides can send frames whenever they want, with minimal overhead (2-10 byte frame header).
After the upgrade, the WebSocket protocol is a series of frames — binary or text messages with a small header. Key frame types:
- Text frames: UTF-8 encoded (typically JSON).
- Binary frames: opaque bytes (use this for protobuf, msgpack, or raw media).
- Ping/Pong: heartbeat frames. Either side can ping; the other must pong. Used to keep the connection alive through idle proxies and detect dead peers.
- Close: a clean shutdown with an optional status code and reason.
The frame header is small (2 bytes minimum) — much cheaper than HTTP's per-request headers. A WebSocket message can be split across multiple frames (fragmentation), useful for streaming large messages or for keeping the connection interactive while a big upload is in flight.
Frames are not request-response. Either side can send any frame at any time. The application has to invent its own message-correlation protocol if it needs to match requests to responses — typically a message ID field in the JSON payload. This is the cost of full-duplex: more flexibility, more responsibility.
Two production details everyone hits:
- Heartbeats are mandatory. Proxies and load balancers (especially on mobile carriers) kill idle connections after 30-60 seconds. Without ping/pong, the client thinks it is connected when it is not. Send a ping every 20-30s; if no pong arrives within a few seconds, reconnect.
- Reconnection is the application's job. WebSocket has no auto-reconnect. The client must detect disconnection (heartbeat timeout, TCP RST) and re-establish, including replaying any messages missed during the gap. Production systems track a 'last event id' so the server can replay missed events on reconnect.
Scaling WebSockets horizontally is the hard part. A single chat server can hold ~50-100k concurrent WebSocket connections (kernel memory per connection, file descriptors, per-connection goroutine/actor). For a million-user app you need many servers, and now you have a routing problem: user A is on server 1, user B is on server 5. When A sends a message to B, how does it get there?
The standard architecture:
- Load balancer: routes incoming WebSocket connections across servers (sticky by user ID hash, or random with pub/sub).
- Pub/sub backbone: a Redis, Kafka, or NATS cluster connecting all the chat servers. When server 1 receives a message for B, it publishes to a topic; server 5 (which holds B's connection) consumes and pushes to B.
- Connection registry: which user is on which server. Often Redis, keyed by user ID. Used to route direct messages.
This is the architecture Slack, Discord, and similar chat systems use. Discord famously runs millions of concurrent WebSockets by sharding users across guild servers and routing messages through a Kafka backbone.
Other scaling concerns:
- Memory per connection: even 50KB per connection adds up at 100k connections = 5GB. Tune your runtime (Go goroutines are cheap; Java threads are not).
- Backpressure: if a client is slow to read, the server's send buffer fills. You must either apply backpressure upstream (slow the producer) or close the slow client.
- Connection draining on deploy: a deploy that kills all connections simultaneously causes a reconnect storm. Roll deploys gradually and let old connections finish.
- CDN/edge termination: WebSockets can be terminated at a CDN edge (Cloudflare supports this), reducing latency to clients and absorbing connection churn.
If the server only needs to push data to the client (live scores, dashboards, notifications), SSE is simpler: it runs over plain HTTP, auto-reconnects, and works through any proxy. If you need true bidirectional communication (chat, gaming, collaborative editing where the client streams lots of data), WebSocket is the right tool. Many teams reach for WebSocket out of habit when SSE would be simpler and more robust.
WebSocket is just a transport — you still need an application-level protocol on top. Three patterns dominate.
JSON-over-text-frames is the default. Simple, debuggable (you can read it in the browser devtools), and integrates with any JSON-aware tool. The cost is verbosity: a 10-byte chat message becomes a 90-byte JSON object plus the WebSocket frame header. Fine for low-frequency chat; painful for 60Hz game updates.
Binary protocols (protobuf, msgpack, CBOR) over binary frames shrink messages dramatically — a protobuf message can be 5-10x smaller than the equivalent JSON. Production real-time systems with high message rates (games, trading) almost always move to binary once the JSON-based protocol stabilizes. The trade-off is debuggability: you need tooling to inspect a binary frame, and schema evolution becomes a concern (protobuf field numbers, backward compatibility).
Standardized subprotocols like STOMP (Simple Text Oriented Messaging Protocol) and WAMP (Web Application Messaging Protocol) layer publish-subscribe semantics on top of WebSocket. STOMP is popular with message brokers (RabbitMQ, ActiveMQ) — clients subscribe to /topic/foo and receive messages. WAMP adds RPC and pub/sub. The advantage is a defined protocol instead of inventing your own; the disadvantage is dependency on a spec that is less widely understood than plain JSON.
Whatever you choose, document it. WebSocket gives you a free-form pipe; the application protocol is where the contracts live. Treat it like any other API: version it, schema-evolve it, and write clients in multiple languages to validate it is portable.
Your app shows a live stock ticker where prices update 10x per second. The client never sends data back. Which transport is most appropriate?
Pick one answer.
You deploy a new version of your chat server, replacing all instances at once. 100k clients reconnect within seconds. What problems will you see, and how should you have deployed?
Pick one answer.
Your users on mobile networks report that their WebSocket connections silently die after ~30 seconds of inactivity. What is happening, and what is the standard fix?
Pick one answer.
Engineering mental model
Mental model. Think of WebSockets — Full-Duplex Real-Time Over TCP as a deliberate boundary in a system. The boundary exists because something becomes harder to manage when everything is done in one place: latency, scale, failure isolation, consistency, cost, or team ownership. The useful question is not “what does WebSockets — Full-Duplex Real-Time Over TCP mean?” but “what pressure makes this boundary worth introducing, and what new failure mode does it create?”
Before choosing WebSockets — Full-Duplex Real-Time Over TCP, name the workload, the critical user path, the dominant bottleneck, the failure you are trying to absorb, and the trade-off you are willing to accept. If you cannot name those five things, the design is probably premature.
// Pseudocode
request = receive()
result = websockets(request)
return result
// Production questions:
// 1. What happens on timeout?
// 2. Can this operation be retried safely?
// 3. What is the bottleneck?Back-of-the-envelope reasoning
Numerical lens: write down traffic, payload size, read/write ratio, peak multiplier and durability target before choosing a component. The numbers should justify the architecture.
Interactive thought experiment: WebSockets — Full-Duplex Real-Time Over TCP
Change the variables below and predict what breaks first in WebSockets — Full-Duplex Real-Time Over TCP. The production lab can later reuse these same inputs.
Change one variable at a time. Predict the failure mode first, then move the slider and see whether your mental model matches the simplified system response.
If you are stuck on WebSockets — Full-Duplex Real-Time Over TCP, start by drawing the request path and marking every network hop, stateful component, queue, cache and failure boundary. Then estimate where the system will saturate.
You increase traffic by 10× in a system using WebSockets — Full-Duplex Real-Time Over TCP. What should you inspect first?
Pick one answer.
Which statement is the safest engineering habit when using WebSockets — Full-Duplex Real-Time Over TCP?
Pick one answer.
You have dashboards for traffic, latency, errors and saturation. You can change the architecture, but every change has operational cost.
Production scenario: your system uses WebSockets — Full-Duplex Real-Time Over TCP, traffic suddenly spikes, and p99 latency doubles. What is your first move?
Interview drill
Answer this without notes: When would you choose WebSockets — Full-Duplex Real-Time Over TCP, and when would you intentionally avoid it? Mention at least one bottleneck it addresses, one failure mode it introduces, and one alternative. Then quantify the workload you are designing for.
A useful engineering lens for WebSockets - Full-Duplex Real-Time Over TCP: define the problem it solves, the simpler design that fails first, the constraint that forces you to introduce this concept, and the new failure modes the concept creates.
Numerical sanity check
Back-of-the-envelope reasoning beats fake precision. State your traffic, payload, concurrency and growth assumptions explicitly, then calculate enough to know whether the current architecture is orders of magnitude away from the target.
Do not optimize for a memorized definition. Reason from the workload and failure mode.
Imagine the simplest version of a system using WebSockets - Full-Duplex Real-Time Over TCP. What breaks first as traffic grows by 10×, and what would you change before reaching 100×?
Pick one answer.
What you gain, what you pay
- +True full-duplex, low-latency bidirectional communication over a single connection.
- +Small per-message overhead (2-10 byte frame header) — much cheaper than HTTP per-message.
- +Works over standard HTTP(S) ports; upgrade mechanism is clean and well-supported.
- +Wide browser support; trivial to use from JavaScript (the WebSocket API).
- +Binary frames enable efficient encoding (protobuf, msgpack) when JSON is too verbose.
- −Stateful connections — the server must hold per-connection state, limiting how many connections one server can hold.
- −No auto-reconnect — the application must detect dead connections and re-establish, including missed-message replay.
- −Routing across multiple servers requires a pub/sub backbone — architectural complexity.
- −Heartbeats are mandatory; without them, mobile connections die silently.
- −Harder to cache than HTTP — proxies and CDNs cannot help with the upgraded channel.
How this breaks in production
- Silent connection death from NAT or proxy idle timeouts — mitigated by application ping/pong.
- Reconnect storm during deploys — mitigated by staggered rollout and client-side jitter.
- Server memory exhaustion from too many connections — mitigated by sharding and connection caps.
- Backpressure stalls when a slow client fills the server's send buffer — mitigated by closing slow clients.
- Load balancer max-connection limits — a single LB might cap at hundreds of thousands of WebSockets.
- Proxy or firewall that strips the Upgrade header — falls back to long polling or fails entirely.
Don't fall into these traps
- •Reaching for WebSocket when SSE would suffice — simpler and more robust for one-way streaming.
- •Not implementing heartbeats — silent connection death on mobile.
- •Not implementing reconnection with replay — users miss messages during network blips.
- •Doing a synchronous deploy across all servers — thundering herd reconnect storm.
- •Sending JSON over WS when binary protobuf would be much smaller for high-frequency messages.
- •Forgetting backpressure — slow clients cause memory bloat and eventual OOM.
- •Treating WS messages as request-response — the protocol is asynchronous; build your own correlation if needed.
Real systems using this
How real systems implement this
- Discord — Millions of concurrent WebSocket connections sharded across guild servers, with Kafka as the cross-server pub/sub backbone. Famous engineering posts detail their scaling journey.
- Slack — Every Slack client holds a long-lived WebSocket for receiving messages, presence, and typing indicators. Connection state is sharded across servers with a pub/sub routing layer.
- Figma — Uses WebSockets for multiplayer collaborative editing. Custom CRDT-based sync over the channel keeps edits conflict-free even with high latency.
Practice saying it out loud
- Q1Design a chat system (Slack/Discord). How do you handle the WebSocket scaling problem?
- Q2WebSocket vs Server-Sent Events vs long polling — when would you choose each?
- Q3Your WebSocket clients on mobile networks silently disconnect after 30 seconds. Diagnose and fix.
- Q4How do you deploy a new version of your chat server without taking 100k users offline?
- Q5How would you handle backpressure in a WebSocket server?
Further reading & references
Core explanations are original NO CAP material. External references are provided for deeper study and standards.
What next?
Mark as understood once the mental model clicks.
Next recommended
Server-Sent Events (SSE)