Publish/Subscribe (Pub/Sub)
Pub/sub is a messaging pattern where producers (publishers) send messages to topics, and consumers (subscribers) receive messages from topics they subscribe to. Unlike a work queue (where each message goes to one consumer), pub/sub delivers each message to ALL subscribers.
How it works
In pub/sub:
- Publishers send messages to a topic (a named channel).
- Subscribers express interest in one or more topics.
- The broker delivers each message to ALL subscribers of that topic.
This is fundamentally different from a work queue, where each message goes to exactly ONE consumer. In pub/sub, every subscriber gets a copy.
Pub/sub is used when:
- Multiple independent consumers need the same event. Example: 'UserCreated' event → email service sends welcome, analytics records signup, recommendation service initializes profile.
- Consumers have different processing speeds. The broker buffers per-subscriber.
- New consumers can be added dynamically without modifying the publisher.
Work queues are used when:
- Each message should be processed once. Example: 'send email' — only one service should send it.
- You want to parallelize work across multiple workers.
- The order of processing matters less than throughput.
Pub/sub is a 'fan-out' pattern: one input, many outputs. The broker duplicates each message to every subscriber. This means total throughput scales with the number of subscribers — if you have 10 subscribers, the broker handles 10x the message volume. This is why pub/sub brokers (Kafka, Pulsar) are designed for massive throughput.
Pub/sub delivery guarantees vary by broker:
- At-most-once: fire-and-forget. Fast, but messages can be lost. Used for metrics/telemetry.
- At-least-once: messages are redelivered if not acknowledged. Most common. Requires idempotent subscribers.
- Exactly-once: rare and expensive. Usually simulated with idempotency keys + transactional consumers.
Most pub/sub systems (Kafka, Google Pub/Sub, SNS) offer at-least-once by default. Subscribers must handle duplicates.
You have 3 services that all need to know when a user signs up: email (send welcome), analytics (record signup), and recommendations (init profile). Which messaging pattern should you use?
Pick one answer.
What is 'fan-out' in the context of pub/sub?
Pick one answer.
Engineering mental model
Mental model. Think of Publish/Subscribe (Pub/Sub) as a deliberate boundary in a system. The boundary exists because something becomes harder to manage when everything is done in one place: latency, scale, failure isolation, consistency, cost, or team ownership. The useful question is not “what does Publish/Subscribe (Pub/Sub) mean?” but “what pressure makes this boundary worth introducing, and what new failure mode does it create?”
Before choosing Publish/Subscribe (Pub/Sub), name the workload, the critical user path, the dominant bottleneck, the failure you are trying to absorb, and the trade-off you are willing to accept. If you cannot name those five things, the design is probably premature.
// Pseudocode
request = receive()
result = pub_sub(request)
return result
// Production questions:
// 1. What happens on timeout?
// 2. Can this operation be retried safely?
// 3. What is the bottleneck?Back-of-the-envelope reasoning
Numerical lens: write down traffic, payload size, read/write ratio, peak multiplier and durability target before choosing a component. The numbers should justify the architecture.
Interactive thought experiment: Publish/Subscribe (Pub/Sub)
Change the variables below and predict what breaks first in Publish/Subscribe (Pub/Sub). The production lab can later reuse these same inputs.
Change one variable at a time. Predict the failure mode first, then move the slider and see whether your mental model matches the simplified system response.
If you are stuck on Publish/Subscribe (Pub/Sub), start by drawing the request path and marking every network hop, stateful component, queue, cache and failure boundary. Then estimate where the system will saturate.
You increase traffic by 10× in a system using Publish/Subscribe (Pub/Sub). What should you inspect first?
Pick one answer.
Which statement is the safest engineering habit when using Publish/Subscribe (Pub/Sub)?
Pick one answer.
You have dashboards for traffic, latency, errors and saturation. You can change the architecture, but every change has operational cost.
Production scenario: your system uses Publish/Subscribe (Pub/Sub), traffic suddenly spikes, and p99 latency doubles. What is your first move?
Interview drill
Answer this without notes: When would you choose Publish/Subscribe (Pub/Sub), and when would you intentionally avoid it? Mention at least one bottleneck it addresses, one failure mode it introduces, and one alternative. Then quantify the workload you are designing for.
For Publish/Subscribe (Pub/Sub), separate producer speed from consumer speed. The key design question is what happens when production temporarily exceeds processing capacity: queue it, shed it, slow producers down, or degrade the feature.
Numerical sanity check
A simple queue sanity check: if producers create 8,000 messages/s and consumers process 6,000 messages/s, backlog grows at roughly 2,000 messages/s until the imbalance is corrected.
Do not optimize for a memorized definition. Reason from the workload and failure mode.
A downstream service slows down while Publish/Subscribe (Pub/Sub) keeps accepting traffic. Would you rather apply backpressure, queue more work, shed load, or degrade? Explain the trade-off.
Pick one answer.
What you gain, what you pay
- +Broadcasts events to all subscribers — one publish, many consumes.
- +Decouples publishers from subscribers — add/remove subscribers without code changes.
- +Each subscriber processes at its own pace — independent scaling.
- +Enables event-driven architecture.
- −Every subscriber gets every message — wasteful if subscribers only care about a subset (use filtering).
- −Higher throughput requirements — broker handles N copies per message.
- −Ordering across subscribers is not guaranteed — each subscriber may process at different speeds.
- −Harder to debug — no single call stack; distributed tracing needed.
How this breaks in production
- Slow subscriber backs up the broker — use per-subscriber queues (Kafka consumer groups, SNS+SQS).
- Duplicate delivery — subscribers must be idempotent.
- Message ordering — within a partition, usually FIFO; across partitions, not guaranteed.
- Schema changes break subscribers — use schema registry and versioning.
Don't fall into these traps
- •Using pub/sub when only one consumer needs each message (use a work queue instead).
- •Forgetting that delivery is at-least-once — subscribers must handle duplicates.
- •Not filtering — if a subscriber only cares about 'high-priority' events, use topic filtering instead of consuming everything.
- •Assuming global ordering — pub/sub usually only guarantees per-partition ordering.
Real systems using this
How real systems implement this
- AWS SNS + SQS — SNS is the pub/sub topic; SQS queues subscribe to it. Each SQS queue is per-consumer, so a slow consumer doesn't block others. This is the standard AWS pattern for fan-out.
- Apache Kafka — Topics with multiple consumer groups. Each group gets a copy of every message (pub/sub). Within a group, messages are distributed across consumers (work queue). This dual mode makes Kafka versatile.
Practice saying it out loud
- Q1What is the difference between pub/sub and a message queue?
- Q2When would you choose pub/sub over a work queue?
- Q3How do you handle slow subscribers in a pub/sub system?
- Q4What delivery guarantees does pub/sub typically provide?
Further reading & references
Core explanations are original NO CAP material. External references are provided for deeper study and standards.
What next?
Mark as understood once the mental model clicks.
Next recommended
Message Queues