Sign in
TodayMapLearnPracticeReview
Library
12 MINadvancedObservabilityNot started

SLO / SLA / SLI

SLI (Service Level Indicator) is a measurement: "99.5% of requests succeeded." SLO (Service Level Objective) is the target: "we aim for 99.9% success over 30 days." SLA (Service Level Agreement) is the contract: "if we miss 99.5%, we refund you." The error budget — the gap between the SLO and 100% — is the controlled resource that lets teams balance reliability against feature velocity.

Why this matters

Without SLOs, "is the service up?" is a vibes question. With SLOs, it's quantitative: we said 99.9% availability, we're at 99.85%, that's an error budget overspend. SLOs let teams decide when to ship features vs stabilize, when to alert vs ignore, and when to escalate vs accept. They turn reliability from an emotional debate ("the site feels slow") into a measurable contract. The error budget is the secret weapon: it gives engineers permission to take risks (deploy, refactor) when there's budget, and forces conservatism when there isn't.

Prerequisites
  • Metrics, Logs, Traces
  • Availability vs Consistency
Related
  • Metrics, Logs, Traces
  • Alerts & Visualization
  • Availability Monitoring
Used in
  • Alerts & Visualization
  • Availability Monitoring
Lesson

How it works

Three terms, often confused:

  • SLI (Service Level Indicator): a measurement. "What fraction of requests succeeded?" "What's the p99 latency?" It's a metric with a definition (numerator, denominator, aggregation window).
  • SLO (Service Level Objective): the target. "99.9% of requests succeed over 30 days." It's an internal goal — what the team commits to.
  • SLA (Service Level Agreement): the contract. "If we miss 99.5% over a month, we refund 10% of your bill." It's a customer-facing promise with consequences.

The relationship is layered: SLA < SLO < 100%. The SLA is the contractual floor — missing it costs money. The SLO is the operational target — usually set 1-2 nines above the SLA so the team has margin. The gap between SLO and 100% is the error budget — the failures the team is allowed to have.

Designing SLIs is harder than it looks. The Google SRE book recommends a structure: "SLI = good events / total events." Examples:

  • Availability: count(requests with status < 500) / count(all requests) over the last 5 minutes.
  • Latency: count(requests with latency < 500ms) / count(all requests) over the last 5 minutes.
  • Freshness: count(records updated in last 5 min) / count(all records).
  • Correctness: count(results matching reference) / count(results).

The trick is choosing what "good" means. Availability alone is weak — a service that returns 500 for 1% of users but 200 for 99% looks fine on average. Latency alone is weak — a service that returns in 1ms but gives wrong answers is failing. Good SLIs combine user-visible metrics: "99% of requests return <500ms with status 200 and a non-empty body."

How to choose an SLO? The trap is picking 100% or 99.999% because it sounds impressive. The right approach:

  1. Measure current performance over a representative period (e.g., 90 days). What's the actual p99 latency? Actual success rate?
  2. Pick an SLO slightly below current performance. If you currently achieve 99.95%, set the SLO at 99.9%. This gives you error budget to use.
  3. Align SLO with user happiness. What latency do users tolerate before they abandon? What error rate do they tolerate before churning?
  4. Re-evaluate quarterly. If you consistently exceed your SLO, raise it. If you consistently miss it, either lower it or invest in reliability.

The cost of more nines is exponential: 99% is one bad day per quarter; 99.9% is one bad day per year; 99.99% requires multi-region active-active with sophisticated automation. Each additional nine roughly doubles operational cost.

Don't set SLOs you can't meet. A 99.99% SLO that you miss monthly is worse than a 99.5% SLO you always hit — it destroys trust and trains teams to ignore alerts.

The error budget policy is what makes SLOs work

An SLO without a policy is just a number. The policy says: when the budget is healthy (green), teams can deploy freely, take risks, ship features. When the budget is burning (yellow), only safe changes. When the budget is exhausted (red), feature work freezes and the team focuses on reliability until the budget recovers. This converts reliability from an argument ("should we ship?") into a rule ("budget is red, freeze"). The rule must be enforced — typically by a release freeze that only the SRE or VP can override. Without enforcement, SLOs are decoration.

The SLA is the contract, and it's deliberately weaker than the SLO. If your SLO is 99.9%, your SLA might be 99.5% — giving you a safety margin where you can miss the SLO (an internal signal) without breaching the SLA (a contractual breach). The SLA specifies consequences: refunds, credits, contract termination. These force the company to take reliability seriously because missing it costs money.

Internal services typically have SLOs but not SLAs — they don't have a customer to refund. But the SLOs still matter: they define what dependencies can rely on. If service A has an SLO of 99.9% and service B depends on A, B can design its own SLO with A's reliability as a floor.

SLAs are usually measured monthly. SLOs are usually measured on a rolling 28-day or 30-day window — long enough to smooth blips, short enough that an outage actually consumes budget.

Check yourself
interview

Your team's SLO is 99.9% availability over 30 days. You're at 99.95%. What does this mean, and what should the team do?

Pick one answer.

Check yourself
core

Why is the SLA usually set lower than the SLO?

Pick one answer.

Check yourself
core

Which SLI is most useful for measuring user-visible latency?

Pick one answer.

Engineering mental model

Mental model. Think of SLO / SLA / SLI as a deliberate boundary in a system. The boundary exists because something becomes harder to manage when everything is done in one place: latency, scale, failure isolation, consistency, cost, or team ownership. The useful question is not “what does SLO / SLA / SLI mean?” but “what pressure makes this boundary worth introducing, and what new failure mode does it create?”

Design lens

Before choosing SLO / SLA / SLI, name the workload, the critical user path, the dominant bottleneck, the failure you are trying to absorb, and the trade-off you are willing to accept. If you cannot name those five things, the design is probably premature.

Original NO CAP systems visual for SLO / SLA / SLI.
Image unavailable. Original NO CAP systems visual for SLO / SLA / SLI.
SLO / SLA / SLI: a compact system-thinking visual.— Original NO CAP visual.
// Pseudocode
request = receive()
result = slo_sla_sli(request)
return result

// Production questions:
// 1. What happens on timeout?
// 2. Can this operation be retried safely?
// 3. What is the bottleneck?
A minimal engineering sketch for reasoning about SLO / SLA / SLI.

Back-of-the-envelope reasoning

Numerical lens: write down traffic, payload size, read/write ratio, peak multiplier and durability target before choosing a component. The numbers should justify the architecture.

Interactive sandboxdeterministic

Interactive thought experiment: SLO / SLA / SLI

Change the variables below and predict what breaks first in SLO / SLA / SLI. The production lab can later reuse these same inputs.

System pressure6%
Try this

Change one variable at a time. Predict the failure mode first, then move the slider and see whether your mental model matches the simplified system response.

Hint

If you are stuck on SLO / SLA / SLI, start by drawing the request path and marking every network hop, stateful component, queue, cache and failure boundary. Then estimate where the system will saturate.

Check yourself
solid

You increase traffic by 10× in a system using SLO / SLA / SLI. What should you inspect first?

Pick one answer.

Check yourself
interview

Which statement is the safest engineering habit when using SLO / SLA / SLI?

Pick one answer.

Try this
interview

You have dashboards for traffic, latency, errors and saturation. You can change the architecture, but every change has operational cost.

Production scenario: your system uses SLO / SLA / SLI, traffic suddenly spikes, and p99 latency doubles. What is your first move?

Interview drill

Answer this without notes: When would you choose SLO / SLA / SLI, and when would you intentionally avoid it? Mention at least one bottleneck it addresses, one failure mode it introduces, and one alternative. Then quantify the workload you are designing for.

Engineering lens

A useful engineering lens for SLO / SLA / SLI: define the problem it solves, the simpler design that fails first, the constraint that forces you to introduce this concept, and the new failure modes the concept creates.

Numerical sanity check

Back-of-the-envelope reasoning beats fake precision. State your traffic, payload, concurrency and growth assumptions explicitly, then calculate enough to know whether the current architecture is orders of magnitude away from the target.

Check yourself
interview

Do not optimize for a memorized definition. Reason from the workload and failure mode.

Imagine the simplest version of a system using SLO / SLA / SLI. What breaks first as traffic grows by 10×, and what would you change before reaching 100×?

Pick one answer.

Trade-offs

What you gain, what you pay

Pros
  • +Quantifies reliability — "is the service up?" becomes a number.
  • +Error budget gives engineers permission to take risks when it's healthy.
  • +Drives prioritization: budget red → freeze features, focus reliability.
  • +SLAs create contractual consequences that fund reliability work.
Cons
  • −Each additional nine roughly doubles operational cost.
  • −SLOs without enforcement are decoration.
  • −Bad SLIs (e.g., averages) give false confidence.
  • −Picking an SLO you can't meet destroys trust and trains teams to ignore alerts.
Failure modes

How this breaks in production

  • SLO set too high (e.g., 100%) — unattainable, always burning budget, no value.
  • SLO set too low — gives false confidence, real user unhappiness not captured.
  • SLI is wrong metric (e.g., CPU usage instead of user-visible latency).
  • Error budget policy not enforced — SLOs ignored when inconvenient.
Common mistakes

Don't fall into these traps

  • •Using average latency instead of percentiles for SLOs.
  • •Setting the SLA equal to the SLO (no safety margin).
  • •Picking SLOs based on vanity rather than user happiness.
  • •Not having an error budget policy — SLOs become passive measurements.
Where you see it

Real systems using this

Google SRE book — formalized SLI/SLO/SLA and error budgets.AWS, GCP, Azure publish SLAs per service with refund terms.Internal platform teams publish SLOs to set expectations with product teams.
Teardowns

How real systems implement this

  • Google SRE — Pioneered SLI/SLO/SLA discipline and error budgets. SRE teams have explicit error budget policies: green → ship features; red → freeze and stabilize.
  • AWS SLAs — Publish contractual SLAs per service (e.g., S3 99.9%, DynamoDB 99.99%) with service credit terms. Internally target much higher (e.g., 99.99% SLO for a 99.9% SLA service).
Interview prompts

Practice saying it out loud

  • Q1Define SLI, SLO, and SLA with examples.
  • Q2What is an error budget, and how is it used to balance reliability vs velocity?
  • Q3How do you choose an SLO target for a new service?
  • Q4Why is the SLA usually lower than the SLO?
Research

Further reading & references

System Design Primer
Open source
ByteByteGo — Scale from zero to millions
ByteByteGo
System Design Tutorial
GeeksforGeeks
System Design Roadmap
roadmap.sh
Observability reference
Reference
Observability reference
Reference
Observability reference
Reference
Google SRE Book
Google

Core explanations are original NO CAP material. External references are provided for deeper study and standards.

What next?

Mark as understood once the mental model clicks.

Next recommended

Metrics, Logs, Traces