Once an agent can act, someone has to decide which actions it may take alone. Get this wrong in one direction and the agent does damage nobody sanctioned. Get it wrong in the other and you have built an expensive autocomplete that interrupts constantly.
The instinct is to treat this as a safety dial — more approvals, more safety. That instinct is wrong in a specific and well-documented way, and understanding why is most of the design.
The autonomy spectrum
| Level | Agent behaviour | Suits |
|---|---|---|
| Suggest | Proposes; a human performs the action | High stakes, low trust, or early rollout |
| Confirm each action | Acts only with per-action approval | Genuinely dangerous environments |
| Confirm consequential | Acts freely; stops for irreversible or costly steps | Most production systems |
| Autonomous with audit | Acts; humans review afterwards | Reversible, bounded, well-measured work |
Most systems belong on the third row, and most first attempts land on the second. The difference between them is not how much you trust the model. It is whether you have done the work to classify actions by consequence.
Classify by consequence, not by category
Diagram, as text
- Proposed action leads to Reversible?
- Reversible? leads to Execute automatically (yes)
- Reversible? leads to Consequence high? (no)
- Consequence high? leads to Execute automatically (no)
- Consequence high? leads to Can it be made reversible? (yes)
- Can it be made reversible? leads to Stage it with a cancellation window (yes)
- Can it be made reversible? leads to Human approval with full context (no)
- Execute automatically leads to Sample for post-hoc review
"Writes need approval, reads do not" is the usual first cut, and it is too coarse. Writing a scratch file and wiring money are both writes.
Five properties actually matter:
Reversibility. Can it be undone, and by whom? This is the dominant question — an action you can trivially revert barely needs a gate.
Blast radius. One record, or the table? One file, or the repository?
Externality. Does anyone outside see it? Sending an email, posting publicly, or calling a partner API cannot be recalled once it lands.
Cost. Money, quota, compute.
Sensitivity. Does it touch personal data, credentials or regulated material?
Score actions on these and the gates place themselves. Editing a file in an isolated workspace: reversible, contained, invisible, free — no gate. Sending a customer email: irreversible, externally visible — always a gate, however confident the agent is.
Make it reversible instead
The most effective way to reduce approvals is not to loosen them. It is to change the actions so they need fewer.
Preview before commit. Have the agent produce a diff, a plan, or a dry-run result. One review of a complete proposal beats twenty per-step confirmations, and the reviewer sees the whole shape.
Stage, then publish. A branch instead of a push. A draft instead of a send. A pending record instead of a live one. The agent works freely inside the staging area; the gate sits at the single boundary where work becomes real.
Group into transactions. Ten related edits reviewed as one change is one decision. Ten separate approvals is ten chances to stop reading.
Isolate the workspace. A container, a scratch database, a separate working directory. Inside it, autonomy is nearly free because mistakes are contained by construction.
The pattern generalises: push the gate to the boundary where consequences become irreversible, and grant broad autonomy behind it.
Give the reviewer enough to decide
An approval prompt that cannot be evaluated is theatre. "Agent wants to run a command. Allow?" gives no basis for a decision, so the human either blocks everything or approves everything.
A usable request states the action precisely, why the agent wants it, what it affects, and whether it can be undone. Denial must also be meaningful: the human needs to be able to reject and steer — "no, use the staging database" — rather than only halt the run. An approval that can only be granted is not a control.
Interruption is as important as approval
Approval gates handle actions you anticipated. Interruption handles everything else.
A human watching an agent go down the wrong path should be able to stop and redirect it without losing the accumulated context. That is a technical requirement, not a UI one: it needs checkpointed loop state and the ability to inject a correction and resume.
Systems lacking it get restarted from scratch instead, which discards useful work and pushes people toward the confirm-each-action mode they were trying to avoid.
Expand autonomy deliberately
Autonomy should widen as reliability is demonstrated, and it should widen explicitly.
Start narrow, measure the intervention rate — how often a human corrects or rejects — and expand where it is low. Scope permissions per tool and per environment rather than globally: an agent trusted to refactor in a sandbox is not thereby trusted to deploy.
The failure to avoid is silent autonomy expansion, where permissions accumulate through convenience — a gate removed to unblock a demo, a default loosened during an incident — until nobody can say what the agent may do. Permissions belong in reviewed configuration, with changes visible.
Accountability
Where an agent acts on a person's behalf, the record needs to answer: what did it do, on whose authority, and what did a human actually see when they approved it.
Log the action, the approving identity, the information presented at approval time, and the outcome. The last two matter more than they look — "a human approved it" is a weak defence if the request did not describe what was really about to happen.
This is also where regulation is heading. Frameworks governing higher-risk automated systems increasingly require meaningful human oversight, and the emphasis is consistently on effective oversight rather than a recorded click. A rubber-stamped approval queue satisfies neither the auditor nor the intent.
Check your understanding
Common mistakes
Approving everything. Produces rubber-stamping, which is worse than no gate.
Gating by category instead of consequence. A scratch file and a wire transfer are both writes.
Approval prompts without enough context. The human cannot evaluate, so they default to yes.
Approval that cannot be redirected. Allow and deny alone is not steering.
No interruption path. People restart runs, lose context, and demand tighter gates.
Silent autonomy expansion. Permissions loosened during incidents and never reviewed.
Treating the approval log as compliance. If the request did not describe the real action, the signature proves little.
Confusing autonomy with capability. An agent good at a task is not automatically trusted to do it unsupervised in production.
Practice
Design the approval policy for an agent handling customer refunds, where most are small and routine and a few are large or unusual. Say what is automatic and what is not.
Write your own answer first, on paper or in a note. Then compare.
Show a hint
Classify by consequence, and design the gate so it is rare enough to be taken seriously.
Show a strong answer
Classify by consequence, not by category. "Refund" is not one action.
- Under £50, order verified, within the returns window, customer's first refund this quarter — automatic. This is the overwhelming majority. Reversible in practice (a wrongly refunded £30 can be recovered or written off), low consequence, high volume. Gating it would be the fatigue mistake.
- £50–500, or a second refund for the same customer this quarter — automatic, with post-hoc review. Executed immediately, sampled and reviewed afterwards. The customer is served at machine speed and the organisation still gets oversight; crucially, review after the fact does not require a human in the latency path.
- Over £500, outside the returns window, or a third refund — approval required. These are rare enough that each one gets genuine attention.
- Anything the agent flags as anomalous — a mismatch between the stated reason and the order, a pattern resembling abuse — approval required regardless of amount.
What the reviewer must be given, or the gate is theatre: the order, the customer's history, the agent's stated reason, the exact amount, and what the agent would do next. A prompt saying "approve refund?" with a number is not enough to decide with.
How to keep the gate meaningful: measure the approval rate. If reviewers approve above roughly 95%, the threshold is too low and should be raised — that number is the signal that the gate has become a formality.
Expand autonomy deliberately: start conservatively, measure the post-hoc review outcomes, and raise the automatic threshold when the data supports it. Loosen on evidence, not on impatience.
An agent is running a long task and the user realises it has misunderstood. Design the interruption mechanism, and say why it is as important as approval.
Write your own answer first, on paper or in a note. Then compare.
Show a hint
Approval is a decision before; interruption is a correction during.
Show a strong answer
Why it matters as much: approval gates handle the actions you anticipated. Interruption handles everything you did not — a misunderstanding of the goal, a change of mind, an approach that is clearly going wrong. Without it, the only options are to watch it finish and clean up afterwards, or kill the process and lose the state, which is why users reach for the browser tab close button.
What the mechanism needs:
- Visible progress as it happens. A user cannot interrupt what they cannot see. Stream the agent's actions and reasoning, not just the final result — this is a prerequisite, not a nicety.
- Interruption that takes effect at the next step boundary, not mid-action. Cancelling halfway through a tool call leaves the external effect ambiguous; stopping between steps leaves a clean, resumable state.
- State preserved on interrupt. The trajectory so far must survive, so the user can redirect — "no, I meant the staging database" — and the agent continues with the work already done rather than starting again.
- Redirection, not just cancellation. The valuable case is "stop and do it differently", and that requires the interrupt to accept a new instruction.
- Clear reporting of what already happened. If it has sent two emails, the user must be told, because those cannot be recalled.
The design principle: an agent that cannot be interrupted must be trusted absolutely, and nothing is. Interruptibility is what makes it acceptable to give the agent more autonomy — the safety comes from being able to stop it, which is cheaper and more general than approving every step in advance.
Key takeaways
- Most systems belong at confirm-consequential, not confirm-everything.
- Classify actions by reversibility, blast radius, externality, cost and sensitivity — not by read versus write.
- Approval fatigue is a real safety failure. Frequent gates become reflexive and manufacture accountability without oversight.
- The best way to need fewer approvals is reversibility: preview, stage, group and isolate.
- Put the gate at the irreversibility boundary, and grant broad autonomy behind it.
- Approval requests must carry enough context to deny, and denial must allow steering.
- Interruption and resumption matter as much as approval, and both need checkpointed state.
- Expand autonomy explicitly, scoped per tool and environment, and instrument the gates you keep.
Further reading
- Ben Shneiderman, "Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy" (IJHCI, 2020) — high automation with high human control.
- Lisanne Bainbridge, "Ironies of Automation" (Automatica, 1983) — why automating the routine degrades human oversight of the exceptional.
- Raja Parasuraman and Victor Riley, "Humans and Automation: Use, Misuse, Disuse, Abuse" (Human Factors, 1997) — automation bias and approval fatigue.
- Google SRE, Site Reliability Engineering (O'Reilly, 2016), chapter 17 — testing and gating changes to reliability-critical systems.
- Anthropic, "Building effective agents" (2024) — autonomy expanded on evidence.