A cash machine that is switched on but hands you the wrong balance is available. It is not reliable.
A cash machine that is always correct but offline half the time is reliable. It is not available.
You want both, they are measured differently, and almost every conversation about "uptime" quietly confuses the two.
The two words, precisely
Availability asks: is it responding? It is a percentage of time the system is up and answering requests. Purely about presence.
Reliability asks: is it responding correctly? It is about producing the right result, consistently, without corruption or loss.
The distinction matters because they fail differently and are fixed differently. An unavailable system is obvious — everyone sees an error page and you get paged within minutes. An unreliable system is far more dangerous: it keeps serving traffic while quietly returning wrong answers, and you might not notice for weeks.
The nines, and what they actually cost
Availability is quoted as a percentage. These sound alike and are wildly different in practice.
| Availability | Downtime per year | Downtime per week | Typical of |
|---|---|---|---|
| 99% (two nines) | 3.65 days | 1.68 hours | Internal tools |
| 99.9% (three nines) | 8.77 hours | 10.1 minutes | Standard SaaS |
| 99.99% (four nines) | 52.6 minutes | 1.01 minutes | Serious platforms |
| 99.999% (five nines) | 5.26 minutes | 6.05 seconds | Telecoms, payments |
Read the third column carefully. At three nines you may be entirely offline for ten minutes a week and still hit target — a deploy that goes wrong and takes eight minutes to roll back is survivable.
At five nines you have six seconds a week. That is less time than a server takes to restart. It means no human can be involved in recovery: nobody can read an alert, log in and diagnose within six seconds. Every failure must be detected and repaired automatically.
Why redundancy is the only real answer
There is exactly one durable way to make an unreliable set of parts into a dependable whole: have more than one of everything, so no single failure can take the system down.
Here is the arithmetic that makes this vivid. Suppose one server is available 99% of the time — it is down about 3.65 days a year.
- One server: 99% available.
- Two independent servers, either of which can serve: the system is only down if both are down at once. That is 1% × 1% = 0.01%, so 99.99% available.
- Three servers: 1% × 1% × 1% = 0.0001%, so 99.9999%.
Two mediocre machines produced better availability than one excellent machine. This is the single most important idea in the topic: reliability comes from arrangement, not from buying better parts.
Diagram, as text
- Users leads to Load Balancer
- Load Balancer leads to Server 1
- Load Balancer leads to Server 2
- Load Balancer leads to Server 3 (takes over if 1 and 2 fail)
The catch: independence
That multiplication only works if failures are independent. In reality they often are not.
If all three servers sit in the same rack, one power supply failure takes out all three at once, and your 99.9999% evaporates. If they all depend on the same database, the database's availability caps the whole system regardless of how many app servers you run.
This is why serious systems spread redundancy across failure domains — different racks, different availability zones, different regions. You are not just buying more machines; you are buying machines that fail for different reasons.
Redundancy is why. Move the slider to change how reliable one machine is, and watch what a second and third buy you.
The first replica is transformative and the third is nearly pointless — the curve flattens hard. That is the shape behind every "two is enough, four is theatre" argument you will have about redundancy budgets.
Single points of failure
A single point of failure is any component whose death stops the whole system. Finding them is a simple, mechanical exercise, and it is one of the highest-value things you can do in a design review.
Point at each box in your diagram and ask: if I delete this right now, does the system still work?
In the earlier scaling lesson, the load balancer itself is the obvious one. It spreads traffic across three servers, so servers are redundant — but if the load balancer dies, no traffic reaches any of them. Real deployments run at least two, with automatic failover.
Measuring reliability: MTBF and MTTR
Two numbers describe how a system behaves around failures.
- MTBF (mean time between failures): how long it typically runs before breaking.
- MTTR (mean time to recovery): how long it typically takes to come back.
Availability is essentially MTBF ÷ (MTBF + MTTR).
The useful insight is that you can improve availability by attacking either number, and lowering MTTR is usually far cheaper. Preventing all failures is impossible; recovering in 30 seconds instead of 30 minutes is an engineering problem you can actually solve — with health checks, automatic restarts, quick rollbacks and good alerting.
Mature teams therefore stop chasing "never break" and start chasing "break briefly and recover automatically".
SLA, SLO and SLI
Three acronyms that sound identical and are not. Read them from the inside out.
- SLI (indicator): the actual measurement. "99.95% of requests succeeded this month." A number you observe.
- SLO (objective): your internal target. "We aim for 99.9%." A goal you set for the team.
- SLA (agreement): the contractual promise to customers, with penalties. "If we drop below 99.5%, you get money back."
They are deliberately ordered: your SLA should be looser than your SLO, which is measured by your SLI. If you promise customers exactly what you target internally, then the moment you miss your goal you also owe refunds. The gap between SLO and SLA is your safety margin.
Check your understanding
Common mistakes
Chasing more nines than the product needs. Five nines for an internal dashboard is an enormous cost for no benefit. Ask what an outage actually costs before setting a target.
Redundancy that shares a failure domain. Three servers in one rack, or three app servers on one database, do not give you the multiplication above. Ask what they have in common, because that is what will take them all down together.
Forgetting the load balancer, the DNS, the config service. Teams carefully make app servers redundant and leave a dozen other single points of failure in place.
Measuring only availability. Uptime dashboards go green while the service returns wrong data. Monitor correctness — error rates, data integrity checks — not just whether the process is running.
Practice
Your product promises 99.9% availability. A stakeholder asks to move to 99.999%. Work out what each target permits in downtime, then explain in business terms what the change would actually require.
Write your own answer first, on paper or in a note. Then compare.
Show a hint
Convert the percentage to minutes before discussing anything else.
Show a strong answer
The budgets, per year:
- 99.9% = 0.1% of 525,600 minutes = ~525 minutes, about 8.7 hours
- 99.999% = 0.001% = ~5.3 minutes
What that means in practice. 8.7 hours a year accommodates a monthly maintenance window, a slow rollback, and a human being paged at 3 a.m. who takes twenty minutes to wake up and diagnose. 5.3 minutes a year does not accommodate a human at all — the entire budget is smaller than the time it takes to read an alert and open a laptop.
So the change is not a configuration setting; it requires:
- Automated detection and failover, because any manual step exceeds the budget.
- Multi-region redundancy with no shared failure domain, because one datacentre event costs more than five minutes.
- Every dependency at the same level or higher — you cannot be more available than the least available thing on your critical path.
- Deployment that cannot cause an outage: progressive rollout, automatic rollback, and no schema change that requires downtime.
The honest framing for the stakeholder: what does one hour of downtime cost us? If the answer is less than the several engineer-years the change implies, 99.9% is the correct target and the conversation should end there.
Draw the failure domains for this system and name every single point of failure: users hit a DNS name, which resolves to one load balancer, which fans out to three app servers in two availability zones, all reading from one primary database with a replica in a third zone.
Write your own answer first, on paper or in a note. Then compare.
Show a hint
Follow one request from the user's keyboard to the data and back, asking at each hop: if this alone dies, is the system down?
Show a strong answer
Walking the path, the single points of failure are:
- DNS itself. One provider, one zone file. If resolution fails, nothing else matters and the failure is invisible to your own monitoring, which usually connects by IP.
- The one load balancer. Three app servers behind a single balancer means the balancer's availability is the system's ceiling. This is the most commonly overlooked one, because teams count the servers and stop.
- The primary database. A replica in a third zone protects the data, but if writes only go to the primary and promotion is manual, a primary failure is a write outage lasting as long as a human takes to notice and promote.
- Anything the app servers share that was not drawn: the configuration service, the secrets store, the container registry they pull from on restart, and the certificate that expires on the same day for all of them.
What redundancy actually exists: only the app tier, and only across two zones — so a two-zone regional event still takes it out.
The fix ordering follows blast radius: a second load balancer (or a managed one with its own redundancy) first, automated database failover second, a third zone for app servers third. The lesson is that counting the redundant components tells you nothing; you have to name what they share.
Key takeaways
- Availability is "is it up"; reliability is "is it right". Silent unreliability is usually the more dangerous failure because nothing alarms.
- Each additional nine costs roughly ten times more. Five nines means six seconds of downtime a week, which rules out human involvement in recovery.
- Redundancy multiplies availability: two independent 99% servers give 99.99%. Reliability comes from arrangement, not from better parts.
- The multiplication only holds if failures are independent — spread across racks, zones and regions.
- Improving MTTR is usually cheaper than improving MTBF. Aim to recover fast rather than never fail.
- Keep your SLA looser than your SLO, so missing an internal target does not immediately breach a contract.
Next we look at what happens when the network between your redundant machines fails — and the theorem that says you must then give something up.
Further reading
- Google SRE, Site Reliability Engineering (O'Reilly, 2016), chapters 3 and 4 — error budgets, and the argument that 100% is the wrong target.
- Google SRE, The Site Reliability Workbook (O'Reilly, 2018), chapter 2 — turning SLIs into SLOs a team can act on.
- Jim Gray, "Why Do Computers Stop and What Can Be Done About It?" (Tandem Technical Report 85.7, 1985) — the foundational study of failure modes and MTTR.
- Nassim Nicholas Taleb, Antifragile (Random House, 2012) — correlated failure and hidden shared dependencies, in general terms.
- Richard Cook, "How Complex Systems Fail" (1998) — a two-page account of why redundancy alone does not produce reliability.