Error Handling

The happy path is the part of a function you can hold in your head. The network succeeds, the file exists, the user types a number. It is also the part that takes the least time to write.

Everything else — the timeout, the malformed row, the half-finished transaction — is where the design work actually lives. Error handling is not defensive clutter bolted onto working code. It is the second interface your function offers, and callers depend on it exactly as much as they depend on the return value.

Fail fast, at the boundary

The cheapest error to handle is the one you reject before doing any work.

def place_order(price, quantity, tax_rate):
    subtotal = price * quantity
    tax = subtotal * tax_rate
    charge_card(subtotal + tax)
    if price < 0:                      # too late
        raise ValueError("invalid price")

The card has already been charged. Validating first is not merely tidier — it is the difference between a rejected request and a corrupted one:

def place_order(price, quantity, tax_rate):
    if price < 0:
        raise ValueError("price cannot be negative")
    if quantity <= 0:
        raise ValueError("quantity must be positive")
    if not 0 <= tax_rate <= 1:
        raise ValueError("tax_rate must be between 0 and 1")
    ...

The discipline generalises: validate at the edge of your system, then trust the data inside it. Parse the request once at the boundary, convert it into a type that cannot be invalid, and every function downstream stops re-checking. Scattering the same null check through fifteen layers is what happens when nobody owns the boundary.

Exceptions or error values

Two families of mechanism, and the choice is about who is expected to handle the failure.

The useful test: is this failure part of the normal range of outcomes? A user submitting a duplicate email is expected — model it as a result the caller must handle. A disk becoming unreadable mid-write is not — let it propagate.

Getting this backwards is common in both directions. Using exceptions for ordinary validation turns control flow into invisible jumps. Returning None for a genuine system failure invites callers to ignore it and crash later with a confusing message.

Catch only what you can act on

A try block should wrap the smallest amount of code that can fail, and catch the narrowest type you know how to handle.

try:
    config = json.loads(path.read_text())
except FileNotFoundError:
    config = DEFAULTS                     # a real recovery
except json.JSONDecodeError:
    raise ConfigError(f"{path} is not valid JSON")   # add context, re-raise

Two anti-patterns dominate real codebases.

Swallowing. except Exception: pass deletes the evidence. The email is never sent, nothing is logged, and the bug is discovered by a customer months later. If you catch it, log it or re-raise it — never neither.

Catching too broadly. A bare except Exception around fifty lines also catches the typo in a variable name, turning a five-second fix into a mystery. Broad catches belong in exactly one place: a top-level handler that logs the failure and returns a generic error to the caller.

When you cannot handle a failure but can explain it, wrap and re-raise. Preserve the original — raise ConfigError(...) from err in Python, %w in Go — or you throw away the stack trace that tells you where it started.

Leave the system in a valid state

The dangerous failures are not the ones that stop work. They are the ones that stop it halfway.

def transfer(source, destination, amount):
    source.debit(amount)
    notify(source.owner)        # raises: the mail server is down
    destination.credit(amount)  # never runs — the money has vanished

Money left the first account and never arrived at the second. No exception surfaces to the user as "your funds are gone", so nobody investigates.

Two rules prevent this:

Make the change atomic. Wrap the state changes in a database transaction so a failure rolls back all of them. Cross-service work needs an explicit pattern — sagas with compensating actions — because there is no shared transaction to roll back.

Do side effects last, outside the transaction. Emails, webhooks and push notifications cannot be rolled back. Commit the state change first, then queue the notification, so a mail outage cannot corrupt your data.

Degrade instead of collapsing

Not every dependency deserves the power to take the page down. Separate what is essential from what is decoration.

If recommendations fail, a product page should render without them. If the reviews service fails, show the product and a quiet placeholder. The judgement is per feature: can a user still accomplish the core task? If yes, catch the failure, log it, and return an empty result.

The failure mode this creates is a slow dependency rather than a broken one. A recommendation service taking 30 seconds to time out holds the whole page hostage, so degradation needs two companions:

Timeouts on every external call. A call with no timeout inherits the timeout of whatever is below it, which may be none. Set one explicitly at every boundary.

A circuit breaker so you stop calling a service that is clearly down. The pattern is a small state machine:

CLOSED     → calls pass through; count consecutive failures
             failures exceed threshold → OPEN

OPEN       → fail immediately without calling; start a cooldown timer
             cooldown elapses → HALF_OPEN

HALF_OPEN  → let one trial call through
             success → CLOSED     failure → OPEN

Failing instantly while OPEN is the point: it frees the threads that would have been blocked waiting, and it gives the struggling service room to recover instead of hammering it with retries.

Retry only what is safe to retry

Retrying a failed call is the obvious response and the easiest way to make an outage worse.

Only retry transient failures. A 500, a timeout or a connection reset may succeed on a second attempt. A 400 or a 404 will fail identically every time; retrying wastes capacity and delays the error the caller needs.

Only retry idempotent operations, or supply an idempotency key so the server can recognise the repeat. Otherwise a lost response turns one payment into two.

Back off exponentially, with jitter. Retrying immediately, in lockstep, from a thousand clients is a self-inflicted denial of service — a thundering herd that arrives precisely when the service is weakest. Randomising the delay spreads the load.

Messages for users, detail for logs

One failure produces two audiences with opposite needs.

The user needs to know what happened and what to do: "That card was declined. Try another payment method." Stack traces and SQL fragments are noise to them and a map of your internals to an attacker.

The log needs everything: the exception type, the stack trace, the request or correlation ID, and the inputs that matter. Include an identifier in the user-facing message so a support ticket maps to a specific log line.

Check your understanding

Common mistakes

Swallowing exceptions. except Exception: pass is the single most expensive line in this lesson. Log it or re-raise it.

Losing the original error. Catching and throwing a fresh exception without chaining discards the stack trace that identifies the real cause.

Using exceptions for control flow. Throwing to exit a loop, or to signal "not found" on a lookup that misses half the time, hides the flow and costs more than a return value.

One generic error type everywhere. If everything raises Exception, callers cannot distinguish a recoverable case from a fatal one and end up catching all or nothing.

Error messages without context. "Operation failed" is unactionable. Say which operation, on which entity, with which value.

No timeout on external calls. The default in most HTTP clients is to wait indefinitely, which converts a slow dependency into a full outage.

Practice

Key takeaways

• Error handling is the second interface your function offers, not an afterthought.
• Validate at the boundary, then trust the data inside — ideally by parsing into types that cannot be invalid.
• Use exceptions for the genuinely exceptional and returned errors for expected outcomes the caller must decide about.
• Catch the narrowest type you can act on; never swallow, and chain when you re-raise so the original cause survives.
• The dangerous failure is the half-finished one. Make state changes atomic and push unrollbackable side effects outside the transaction.
• Degrade rather than collapse, backed by explicit timeouts and a circuit breaker so a slow dependency cannot hold the page hostage.
• Retry only transient failures on idempotent operations, with exponential backoff and jitter.
• Give users an actionable message, logs the full detail, and share an identifier that links the two.

Further reading

• RFC 9457, Problem Details for HTTP APIs — a standard shape for machine-readable error responses.
• Michael Nygard, Release It!, second edition (Pragmatic Bookshelf, 2018) — failing fast and degrading deliberately.
• Andrew Gerrand, "Errors are values" (The Go Blog, 2015) — the case for errors as ordinary return values.
• Joe Armstrong, Making Reliable Distributed Systems in the Presence of Software Errors (doctoral thesis, 2003) — let it crash, and supervise.
• Google SRE, Site Reliability Engineering (O'Reilly, 2016), chapter 22 — retries as an amplifier.