Five Causes Behind Most Outages: An Opinionated Taxonomy

Read enough postmortems and something odd happens. The stories stay distinct — different services, different teams, genuinely different bugs — but the shapes start repeating, and after a while you can often guess the shape from the first two sentences.

What follows is a taxonomy built from that pattern-matching. It is opinionated, it comes from experience rather than any study, and I have no numbers to offer you about how much it covers. Treat it as a set of questions worth asking in order, not as a finding.

Something changed

The overwhelming first suspect, and it stays the first suspect no matter how many times it turns out to be innocent.

A deploy, obviously — but also a config push, a feature flag turned on for the next percentile of users, a certificate rotation, a dependency version bumped by an automated pull request, a DNS record, an IAM policy, a database parameter someone adjusted to fix a different problem last week. Systems in steady state tend to stay in steady state. Change is what moves them.

The tell is correlation in time, and the reason this class is worth checking first is that it is the cheapest to disprove. If nothing changed in the window, you have eliminated a large slice of possibility in about ninety seconds. If something did, you probably have your answer, and you have a fast remedy that does not require understanding the mechanism.

The trap is a narrow definition of change. Teams check their own deploy log, find it empty, and move on — while the change was in a config repository, a third-party API's behaviour, or a platform team's cluster upgrade that nobody thought worth announcing.

A dependency degraded rather than failed

This one is harder than outright failure, and it is more common.

When a dependency dies cleanly, your circuit breaker trips, your fallback runs, and your alerting fires on an error rate that is unmistakable. When it merely gets slow, none of that happens. Health checks still pass, because the health endpoint does not touch the slow path. The error rate stays at zero. Your threads sit blocked on responses that are still, technically, arriving.

Partial degradation is where most cascades begin, because the systems in front of it keep accepting work they cannot complete, and the backlog propagates upstream faster than anyone can diagnose it.

The tell is latency rising while errors stay flat, and saturation showing up in a service with no fault of its own. The countermeasures are unglamorous and mostly consist of admitting that timeouts must be aggressive and that a health check which does not exercise the real dependency is decoration.

Something ran out

The slow-motion class. Something that only ever grows crossed a limit.

Disk. Memory. File descriptors. Connection pool slots. Thread pool workers. Inodes. Ephemeral ports. Log volume. Rows in a table with a 32-bit primary key. Certificate validity, which is a countdown from the day it is issued. Cloud quota you did not know was per-region.

What makes this class distinctive is the curve: a long, boring, entirely predictable ramp followed by a cliff. Every one of these is visible weeks in advance and almost none of them are alerted on, because the graph is uninteresting right up until the moment it is the outage.

The tell is that the incident started with no change at all — the fatal one usually did not require anyone to do anything. Ask what has been growing.

A loop closed

The most dangerous class, and the only one where removing the original cause does not end the incident.

Retries generating load in proportion to failures. Health checks removing nodes, concentrating traffic on the survivors, causing them to fail their health checks too. Autoscaling reacting to a metric that its own scaling action moves. Clients reconnecting in unison after a restart. A cache stampede refilling from an origin that is falling over because of the stampede.

In every case the system contains a path where load causes failure and failure causes load. Once that gain passes one, the incident sustains itself, and the original trigger becomes irrelevant — often it has quietly healed while you are still down.

The tell is unmistakable if you know to look for it: the system will not recover even after the trigger is gone, or it recovers and immediately collapses again the moment traffic returns. The exit is nearly always the same regardless of which loop it is — shed load hard at the edge, let the system come up under a fraction of real traffic, then ramp. Fixing the loop comes afterwards.

The distribution changed, not the volume

The last one is the subtlest, because your capacity planning was not wrong. It was answering a different question.

You sized for aggregate throughput. What arrived was the same throughput shaped differently: one tenant with a hundred times the normal object count, one celebrity account fanning out to millions of followers, one key on one shard taking a share of traffic no rebalancing anticipated, one customer's integration sending a query that no index covers, one batch job scanning a range that was tiny when the feature launched.

The tell is that your aggregate dashboards look completely normal while one shard, one partition, one queue or one instance is on fire. Fleet-wide averages are actively harmful here; they are designed to hide exactly this.

The countermeasure is per-tenant and per-key visibility, plus limits that bound what any single caller can consume — which is the real argument for rate limiting having nothing to do with abuse.

Why the classes matter more than the incidents

Individual postmortems age badly. The specific bug is fixed, that code is rewritten within a year, and the document becomes archaeology.

The classes do not age, and that is what makes them useful during an incident rather than after it. Instead of the paralysing question — what is wrong? — you get five in a fixed order, each of them cheap: What changed? What is slow but not erroring? What has been growing? Is anything feeding itself? Is the load normal in aggregate but concentrated somewhere?

Two of those you can answer in a minute from dashboards you already have. That is worth considerably more at three in the morning than a complete theory of failure.

I would not defend this list as exhaustive, and you should extend it with whatever your own systems keep doing to you. The value is not in the taxonomy being right. It is in having any ordered set of questions at all, so that the first ten minutes of an incident are spent eliminating rather than guessing.