Priority Queue
A priority queue processes messages by priority rather than arrival order. High-priority messages jump the queue ahead of low-priority ones, ensuring that critical workloads — premium users, payment failures, security alerts — get handled first even under sustained load. Implementation can be N separate FIFO queues polled in priority order, a single queue with a priority field and a heap-based consumer, or weighted fair queuing. The pattern trades FIFO simplicity for SLA-aware processing.
How it works
A FIFO queue is fair — every message waits its turn based on arrival time. But fairness isn't always what you want. A payment retry for a paying customer shouldn't wait behind 100,000 free-user image resizes. A security alert shouldn't wait behind a daily digest email. A priority queue solves this: messages are dequeued by priority, not by arrival order.
Three common implementations:
1. Multiple queues, priority-ordered polling. Maintain N queues (e.g., critical, high, normal, low). Consumers poll in priority order: drain critical first, then high, then normal, then low. Simple, works with off-the-shelf queue systems (just N queues). Risk: low-priority queues can starve — never get processed if high-priority is always non-empty. Mitigation: weighted scheduling (process N high-priority for every M low-priority).
2. Single queue with priority field and heap-based consumer. All messages go to one queue, each with a priority field. The consumer uses a priority heap (or sorted structure) to always dequeue the highest priority first. More flexible (continuous priority values), but requires a broker that supports priority natively (RabbitMQ priority queues, ActiveMQ) or custom consumer logic.
3. Weighted fair queuing (WFQ). Each priority gets a weight; consumers process messages proportional to weights. E.g., priority 1 gets 70% of capacity, priority 2 gets 20%, priority 3 gets 10%. Guarantees no queue starves, while still favoring high priority. More complex but fairer than strict priority.
Most practical systems use option 1 (multiple queues, priority-ordered polling) because it works with any message broker and is easy to reason about.
Key design questions:
- How many priority levels? Typically 3-5 (e.g., critical, high, normal, low, background). Too many = hard to manage; too few = no differentiation.
- Strict priority or weighted? Strict = low can starve; weighted = no starvation, less dramatic prioritization.
- Starvation prevention? Aging — bump a message's priority after N minutes in queue. Or weighted scheduling.
- Per-customer or global? Per-customer priority (premium users always jump ahead) is more complex; global priority (critical workloads first) is simpler.
The defining risk of strict priority queues is starvation: low-priority messages may never be processed if higher-priority messages are always arriving. A real example: in a customer support system, if ‘premium’ tickets always jump ahead of ‘free’ tickets, free tickets may sit for hours or days during peak load.
Mitigations:
1. Aging — boost a message's priority after it has waited N minutes. A low-priority message that has been waiting 30 minutes is treated as normal; one waiting 2 hours is treated as high. This guarantees every message eventually gets processed, while still favoring high-priority messages most of the time.
2. Weighted scheduling — allocate consumer capacity proportionally: 70% to critical, 20% to high, 8% to normal, 2% to low. Critical gets most throughput, but low always gets some. No starvation, less aggressive prioritization than strict.
3. Capacity guarantees — guarantee a minimum throughput per priority. E.g., ‘low gets at least 10 messages/second even if critical is backlogged.’ Requires weighted scheduling or dedicated consumers per queue.
4. Bounded backlog — drop or expire messages that have been in queue too long. Acceptable for some workloads (notifications, telemetry), unacceptable for others (payments, orders).
5. Separate consumer pools per priority — instead of one consumer pool polling all queues, run separate pools: 10 consumers for critical, 3 for high, 1 for normal, 1 for low. Critical gets guaranteed capacity; normal and low still progress. Operationally more complex but predictable.
The choice depends on the workload: if low-priority messages can be dropped (telemetry), use bounded backlog. If they can't (orders), use aging or weighted scheduling.
Priority queues are the operational foundation of tiered SLAs. ‘Premium customers get p99 < 1s; free customers get p99 < 10s.’ Without priority queues, all customers see the same degraded latency during load. With them, premium customers' messages jump the queue, and their SLA holds even when the system is saturated with free-tier work. This is how SaaS products offer meaningful paid tiers — the priority queue is the mechanism that backs the SLA contract.
When to use a priority queue:
- SLA differentiation between customers — premium vs free tiers, enterprise vs standard.
- Mixed workloads with different urgency — user-triggered tasks (high) vs background tasks (low).
- Failure recovery — payment retries, message redelivery with high priority so failures don't linger.
- Alerts and notifications — security alerts ahead of marketing emails.
- Triage scenarios — emergency room patients, security incidents, customer escalations.
When NOT to use a priority queue:
- Uniform SLA across all messages. A simple FIFO is simpler and sufficient.
- You only have one priority level. No benefit; adds complexity.
- You can't meaningfully assign priorities at produce time. If you don't know which messages matter, the queue can't either.
- Strict ordering required. Priority queues break FIFO; if you need ordering within an entity, combine with Sequential Convoy (priority + partition).
A common pattern: combine priority queues with Sequential Convoy. Partition by entity ID (for ordering), and have N priority levels within each partition. The consumer processes high-priority messages first, but within a priority, preserves per-entity ordering. This is how ticketing systems handle ‘premium customer ticket must jump the queue but tickets for the same customer must be processed in order.’
You have a customer support system where premium users should get faster response than free users. During peak load, free users' tickets wait for hours. Which pattern, and how do you prevent free users from never being served?
Pick one answer.
What is the starvation problem in strict priority queues, and which mitigation guarantees every message eventually gets processed?
Pick one answer.
Engineering mental model
Mental model. Think of Priority Queue as a deliberate boundary in a system. The boundary exists because something becomes harder to manage when everything is done in one place: latency, scale, failure isolation, consistency, cost, or team ownership. The useful question is not “what does Priority Queue mean?” but “what pressure makes this boundary worth introducing, and what new failure mode does it create?”
Before choosing Priority Queue, name the workload, the critical user path, the dominant bottleneck, the failure you are trying to absorb, and the trade-off you are willing to accept. If you cannot name those five things, the design is probably premature.
message_id = queue.publish({
"type": "priority-queue",
"key": resource_id
})
# Consumer must be safe to retry.Back-of-the-envelope reasoning
Numerical lens: write down traffic, payload size, read/write ratio, peak multiplier and durability target before choosing a component. The numbers should justify the architecture.
Interactive thought experiment: Priority Queue
Change the variables below and predict what breaks first in Priority Queue. The production lab can later reuse these same inputs.
Change one variable at a time. Predict the failure mode first, then move the slider and see whether your mental model matches the simplified system response.
If you are stuck on Priority Queue, start by drawing the request path and marking every network hop, stateful component, queue, cache and failure boundary. Then estimate where the system will saturate.
You increase traffic by 10× in a system using Priority Queue. What should you inspect first?
Pick one answer.
Which statement is the safest engineering habit when using Priority Queue?
Pick one answer.
You have dashboards for traffic, latency, errors and saturation. You can change the architecture, but every change has operational cost.
Production scenario: your system uses Priority Queue, traffic suddenly spikes, and p99 latency doubles. What is your first move?
Interview drill
Answer this without notes: When would you choose Priority Queue, and when would you intentionally avoid it? Mention at least one bottleneck it addresses, one failure mode it introduces, and one alternative. Then quantify the workload you are designing for.
For Priority Queue, separate producer speed from consumer speed. The key design question is what happens when production temporarily exceeds processing capacity: queue it, shed it, slow producers down, or degrade the feature.
Numerical sanity check
A simple queue sanity check: if producers create 8,000 messages/s and consumers process 6,000 messages/s, backlog grows at roughly 2,000 messages/s until the imbalance is corrected.
Do not optimize for a memorized definition. Reason from the workload and failure mode.
A downstream service slows down while Priority Queue keeps accepting traffic. Would you rather apply backpressure, queue more work, shed load, or degrade? Explain the trade-off.
Pick one answer.
What you gain, what you pay
- +SLA differentiation — high-priority messages jump the queue.
- +Critical workloads (payment failures, security alerts) get fast response even under load.
- +Premium customers can be guaranteed better latency than free.
- +Multiple priority levels enable nuanced triage (critical / high / normal / low / background).
- −Starvation risk — low-priority messages may wait indefinitely under sustained high-priority load.
- −More complex than FIFO — multiple queues, priority assignment logic, aging or weighted scheduling.
- −Requires careful priority assignment at produce time — wrong priority defeats the purpose.
- −Ordering within a priority is still FIFO — combine with Sequential Convoy for per-entity ordering.
How this breaks in production
- Starvation — low-priority messages never processed under sustained high-priority load.
- Wrong priority assignment — every message marked critical, defeating differentiation.
- Priority queue hotspots — one priority dominates, others idle.
- Poison message in critical queue blocks subsequent critical messages — needs dead-letter handling per queue.
- Aging bugs — wrong timing or priority bump thresholds cause unexpected re-ordering.
- Weighted scheduling misconfiguration — weights don't match actual load distribution.
Don't fall into these traps
- •Using strict priority without aging or weighted scheduling — low-priority starves.
- •Assigning priority based on producer's intuition rather than objective criteria.
- •Too many priority levels — complexity without differentiation.
- •Not monitoring per-priority queue depth — silent starvation is invisible without metrics.
- •Treating priority as static — load conditions change; priorities should adapt (dynamic priority).
- •Forgetting to dead-letter per priority — a poison message in `critical` blocks the whole critical queue.
Real systems using this
How real systems implement this
- RabbitMQ priority queues — RabbitMQ supports priority queues natively via the `x-max-priority` queue argument. Messages are sorted by priority (1-255); the consumer always receives the highest-priority message first. Used for SLA-differentiated workloads in many on-prem message-driven systems.
- AWS SQS with multiple queues + priority polling — AWS SQS doesn't natively support priority, but the standard pattern is multiple queues (critical.fifo, normal.fifo, low.fifo) polled by a consumer in priority order. Used widely for tiered SLA workloads on AWS.
- Kubernetes Priority Classes — Kubernetes assigns PriorityClass to pods; the scheduler preempts lower-priority pods to schedule higher-priority ones. PriorityClasses range from cluster-autoscaler to system-critical to user-defined. A system-level implementation of the priority queue pattern for pod scheduling.
Practice saying it out loud
- Q1What is a priority queue, and when would you use one over a FIFO queue?
- Q2What is the starvation problem in strict priority queues, and how do you mitigate it?
- Q3How would you implement SLA differentiation for premium vs free customers using priority queues?
- Q4How does a priority queue interact with ordering requirements? How would you combine per-entity ordering with priority?
- Q5What are the trade-offs between strict priority and weighted fair queuing?
Further reading & references
Core explanations are original NO CAP material. External references are provided for deeper study and standards.
What next?
Mark as understood once the mental model clicks.
Next recommended
Queue-Based Load Leveling