On this page
Service Level Objectives (SLOs)
Defining numerical targets for system reliability that balance the need for feature velocity against the risk of user-facing outages.
What it is
A Service Level Objective (SLO) is a precise numerical target for the reliability of a service. It represents the point at which users are statistically likely to be happy with the system’s performance.
SLOs are built upon Service Level Indicators (SLIs), which are the actual metrics being measured (e.g., the percentage of HTTP 200 responses).
A complete SLO defines the SLI, the target percentage, and the time window. For example: “99.9% of all HTTP requests to the Checkout Service must return a successful status code in under 200 milliseconds, measured over a rolling 30-day window.”
Why it matters
Engineers naturally want to build new features (velocity). Operations teams naturally want the system to never change, because change causes outages (reliability). This creates endless friction.
SLOs resolve this conflict using math. By defining an objective of 99.9% reliability, the organization explicitly agrees that 0.1% unreliability is acceptable. This 0.1% is the “Error Budget.”
If a team has a healthy error budget, they are encouraged to push new features, run experiments, and deploy rapidly. If bad deployments consume the error budget and push the service below 99.9%, all feature work halts. The team must freeze deployments and focus exclusively on reliability engineering until the budget recovers. SLOs turn subjective arguments into objective operational rules.
How to implement
1. Define the SLI
Determine what actually matters to the user. Do not measure CPU or RAM. Measure the user journey.
- Availability SLI: The proportion of valid requests served successfully (Good Requests / Total Requests).
- Latency SLI: The proportion of valid requests served faster than a specific threshold (Fast Requests / Total Requests).
2. Set the Target (SLO)
Do not set the target to 100%. 100% reliability is impossible, and attempting to achieve it costs exponentially more money than 99.9% reliability. If your users cannot perceive the difference between 99.9% and 99.99% availability (because their own cellular networks drop packets more frequently than that), do not pay the massive engineering cost to hit 99.99%.
3. Track the Error Budget
If your SLO is 99.9% over a 30-day window, you are allowed 43 minutes of total downtime per month.
- 30 days = 43,200 minutes.
- 0.1% of 43,200 = 43.2 minutes of Error Budget.
Connect your monitoring tools to this budget. When an outage occurs, subtract the downtime from the budget.
Common mistakes
Unrealistic Targets
Business stakeholders often demand “five nines” (99.999% availability). Five nines allows for exactly 5 minutes of downtime per year. This means no manual deployments, instantly redundant global database failovers, and absolute perfection. If the business is not willing to spend millions of dollars re-architecting the system to achieve this, the SLO is a fantasy and will be ignored by engineering.
Punitive Error Budgets
If an error budget is depleted, the reaction must be systemic, not punitive. The team stops feature work to fix technical debt, not because they are being punished, but because the math dictates the system is too fragile to sustain further change. If management uses depleted error budgets to punish developers, developers will simply change the SLI definitions to hide failures and protect themselves.
Third-Party Dependencies
You cannot set an SLO of 99.99% if your service relies on a third-party payment gateway that only guarantees 99.9% availability. Your service’s mathematical maximum reliability is strictly capped by the reliability of its critical dependencies.
Verification
To verify your SLOs are actionable, implement “Burn Rate Alerting.”
Instead of alerting when the error rate spikes momentarily (which causes alert fatigue), alert when the error budget is burning so fast that it will be depleted in a few hours.
A slow burn (consuming the budget over 30 days) is fine; that is normal operational friction. A fast burn (consuming 10% of the monthly budget in a single hour) requires an immediate page to the on-call engineer, because a catastrophic failure is actively occurring.
Verify your alerting configuration outputs burn rate metrics:
job:slo_errors_per_request:ratio_rate1h{job="checkout"} > (14.4 * 0.001)