Everything so far has been a single model call: assemble a prompt, get an answer, return it.
Some tasks cannot work that way. "Find why checkout latency rose yesterday and open a ticket" requires querying metrics, reading deploy history, correlating them, deciding what happened, and calling an API — where each step depends on what the previous one found. You cannot write that prompt in advance, because you do not know what step three is until step two runs.
An agent is a model in a loop with tools, deciding its own next action.
The loop
Strip away the terminology and every agent is this:
Diagram, as text
- Goal leads to Model decides next action
- Model decides next action leads to Action or finished?
- Action or finished? leads to Execute tool (tool call)
- Execute tool leads to Observation
- Observation leads to Model decides next action
- Action or finished? leads to Done (answer)
The model reasons about the goal, chooses a tool, sees the result, and reasons again with that result added to its context. It repeats until it decides it is finished.
That is the whole idea. The engineering is in controlling it.
Patterns, in increasing order of ambition
Chains are not agents. A fixed sequence — retrieve, then summarise, then classify — where you decided the steps. Deterministic, debuggable, and correct for most problems. Start here.
Routing picks one of several fixed paths based on the input. Still deterministic once the branch is chosen.
ReAct is the basic agent loop above: reason, act, observe, repeat. Flexible, and its control flow is decided at runtime by the model.
Plan-and-execute asks the model to write a plan first, then executes the steps. The plan is inspectable before anything runs — a genuine operational advantage — and it can be re-planned when a step fails.
Multi-agent splits work across specialists with an orchestrator delegating: a researcher, a writer, a reviewer. Appealing in principle, expensive in practice, and usually attempted too early.
What makes agents hard
Not the loop. The control.
They do not know when to stop. An agent can loop indefinitely, retrying a failing tool or circling between two actions. You need a hard cap on steps, a wall-clock timeout and a token budget — enforced by your code, never by asking the model politely.
Cost is unbounded by default. Each step is a model call carrying the entire accumulated context, so a ten-step task is not ten times a single call — it is more, because context grows with every observation. Set a budget per task and abort when it is exhausted.
Errors compound. If each step is 95% reliable, ten sequential steps give roughly 60% end-to-end. Long agent trajectories are unreliable for arithmetic reasons, regardless of model quality. Fewer steps is a correctness strategy.
Failures are silent. A wrong tool argument returns an error the model reads and reasons around — sometimes recovering, sometimes inventing a workaround that does damage. Nothing throws.
They take real actions. This is the qualitative break from everything earlier in the section. A RAG system that is wrong produces bad text. An agent that is wrong sends an email, deletes a record, or issues a refund.
Controlling the blast radius
Because actions are real, the safety design matters more than the orchestration pattern.
Separate read tools from write tools. Reads can be liberal; writes need constraints. Many production agents are read-only by design and hand a proposed action to a human.
Require confirmation for consequential actions. Anything irreversible or externally visible — sending, paying, deleting, publishing — should pause for approval. This is a product decision as much as a technical one.
Make every tool idempotent, and pass an idempotency key. Agents retry. Without idempotency, a retried refund is a second refund — exactly the reasoning from the API design lesson.
Constrain at the tool boundary, not in the prompt. "Only refund up to ₹500" in a system prompt is a suggestion. A check inside the refund tool is a rule. Never rely on instructions for authorisation.
Scope credentials to the agent's job. If it can only read the orders table, a confused or manipulated agent cannot drop it. Least privilege, applied to a component that decides its own actions.
Multi-agent, honestly
The pattern is a single orchestrator delegating to specialists, each with its own tools and prompt.
The genuine benefits: focused prompts and small tool sets per agent, which improves tool selection; parallelism when subtasks are independent; and clearer ownership boundaries in a large system.
The costs are substantial and usually underestimated. Every delegation is more model calls. Context must be passed between agents and is frequently lost at the boundary. Debugging spans several trajectories. And failure modes multiply — agents can loop between each other, or an orchestrator can misroute persistently.
A single agent with a well-chosen tool set beats a multi-agent system for most problems. Reach for multi-agent when subtasks are genuinely independent and parallelisable, or when specialists need such different tools and instructions that one prompt cannot serve both.
The state problem
An agent trajectory is a long-running, stateful process — which makes it a distributed systems problem, not a prompting one.
Where does state live if the process crashes at step seven? Can a user resume tomorrow? What happens if two requests run for the same user concurrently?
Treat a trajectory like a workflow: persist each step, make steps replayable, and design for resumption. The saga reasoning from the distributed transactions lesson applies directly — including compensating actions for steps that already took effect when a later step fails.
Check your understanding
Common mistakes
Building an agent when a chain would do. The most common and most expensive error.
No step, time or token limit. An unbounded loop against a paid API is a bill and an incident.
Authorisation in the prompt. Enforce in the tool.
Non-idempotent tools. Agents retry; duplicated side effects follow.
Multi-agent too early. Costs multiply before benefits appear.
No trajectory persistence. A crash loses the whole task with no way to resume.
Evaluating only the final answer. A correct answer reached by a wasteful or dangerous path is a problem you cannot see without inspecting the trajectory.
Practice
Design the blast-radius controls for an agent that can read a codebase, run tests, edit files and open pull requests. Be specific about what it may do without approval.
Write your own answer first, on paper or in a note. Then compare.
Show a hint
Rank the tools by whether the action is reversible and who sees the result.
Show a strong answer
Classify each tool by reversibility, because that is what determines the control.
- Read files, search, run tests — no approval, no limit beyond cost. Read-only, reversible by definition, and requiring approval here destroys the value of the agent. Do bound them: a test suite that runs for an hour needs a timeout.
- Edit files — permitted, but only inside a workspace it owns. The agent works on a branch in a checkout it cannot escape, never on the main branch and never on the developer's working tree. The control is the sandbox, not a permission prompt: the action is freely reversible because nothing outside the branch changed.
- Open a pull request — permitted, because a PR is a proposal. It changes nothing until a human merges it. This is the natural approval point, and it is one that already exists in the workflow rather than a new gate invented for the agent.
- Merge, push to main, deploy, delete branches, comment on other people's PRs — never. These are either irreversible or externally visible. The last one matters more than it looks: an agent that comments publicly is acting in your name to other people.
Controls that apply regardless of tool:
- No credentials beyond the workspace. No production database access, no cloud credentials, no ability to reach internal services. If the agent is compromised by content in the repository it reads — which is a live risk, since source files can contain instructions — this is what bounds the damage.
- No network access from the test runner except a package mirror, so a malicious dependency cannot exfiltrate.
- Every action logged with the reasoning that led to it, because debugging an agent means reconstructing why, not just what.
- Iteration, time and token caps, with the run abandoned and reported rather than silently continuing.
The principle: let it act freely where actions are reversible and contained, and stop it entirely where they are not. Approval prompts on every step train people to click yes.
An agent completes 70% of tasks successfully. Product wants 95%. Describe how you would find out whether that is achievable and what would get you there.
Write your own answer first, on paper or in a note. Then compare.
Show a hint
Aggregate success rate hides several different failures with different fixes.
Show a strong answer
The number is useless until it is broken down, because 70% is not one problem.
Categorise the 30% of failures by cause, from logged trajectories:
- Wrong tool chosen, or tool called with wrong arguments. Usually the largest bucket, and usually fixable with better tool descriptions, clearer parameter names and a few examples. Cheap and high-yield.
- Tool failed and the agent did not recover. A transient error the agent treated as final, or a retry loop it never escaped. This is error-handling design, not model capability.
- The task was ambiguous. The agent did something defensible that was not what the user meant. This is not a model failure; it is a missing clarification step, and forcing a question when confidence is low will move it.
- The model genuinely could not do it — reasoning beyond its capability. This is the only bucket where a better model helps.
- The task was impossible — the information did not exist, the permission was absent. Success here means failing clearly, and these should be excluded from the denominator or counted separately.
What I would tell product about 95%:
The first three buckets are engineering and usually movable. The fourth is bounded by the model. If ambiguous and impossible tasks together are more than 5% of the distribution, 95% is not achievable at all with this definition of success — and the honest response is to change the definition: measure success or correct escalation, where an agent that recognises it cannot proceed and hands to a human counts as a success rather than a failure.
That reframing is usually both achievable and more valuable, because an agent that fails loudly at 5% is safe to deploy and one that fails silently at 5% is not.
Key takeaways
- An agent is a model in a loop with tools, choosing its next action at runtime.
- Prefer the least agentic option: chains and routing are deterministic and correct for most tasks.
- Agents need hard limits on steps, time and tokens, enforced in code — they do not reliably stop on their own.
- Errors compound multiplicatively, so shorter trajectories are more reliable, not just cheaper.
- The qualitative risk is that agents take real actions — separate reads from writes, confirm consequential ones, and make every tool idempotent.
- Enforce limits at the tool boundary, never in the prompt.
- Multi-agent multiplies cost and failure modes; a single agent with a good tool set usually wins.
- A trajectory is a long-running stateful workflow — persist steps and design for resumption and compensation.
Further reading
- Shunyu Yao et al., "ReAct: Synergizing Reasoning and Acting in Language Models" (ICLR 2023).
- Timo Schick et al., "Toolformer: Language Models Can Teach Themselves to Use Tools" (NeurIPS 2023).
- Noah Shinn et al., "Reflexion: Language Agents with Verbal Reinforcement Learning" (NeurIPS 2023).
- Anthropic, "Building effective agents" (2024) — the argument for simple loops and good tools over multi-agent architectures.
- Michael Nygard, Release It!, second edition (Pragmatic Bookshelf, 2018) — bounded resources, which is what an agent loop needs most.