You have an agent that works. The obvious next move is more agents — a planner, a researcher, a reviewer, each specialised.
This is the topic where that intuition is most often wrong, and it is worth knowing that credible production teams have published directly opposing conclusions about it. One reported that an orchestrator with parallel subagents substantially outperformed a single agent on open-ended research. Another published a piece titled, in effect, don't build multi-agent systems, arguing they are fragile and that you should stay single-threaded.
Both were right about their own workload. The reconciliation that has emerged is narrower than the hype: multi-agent is a way to allocate context and compute, not a way to buy intelligence.
Control for tokens before you believe anything
The single most useful discipline in this area is refusing to compare systems that spent different amounts.
Multi-agent systems consume dramatically more tokens than a single agent — one published production system reported roughly fifteen times the tokens of a plain chat interaction. If you compare that against a single agent running at default effort, you are not comparing architectures. You are comparing budgets.
When researchers began matching reasoning-token budgets between single-agent and multi-agent setups, much of the reported multi-agent advantage disappeared. The team behind one of the most-cited multi-agent results found, in their own analysis, that token usage alone explained around 80% of the performance variance on their benchmark.
Every handoff loses information
There is a clean theoretical reason to be suspicious of decomposition. Information theory's data-processing inequality says that passing data through an additional processing step cannot add information — it can only preserve or lose it. Every agent-to-agent handoff is exactly such a step, and it is a lossy one: rich internal reasoning and tool results get serialised into a short text message.
The practical version is easier to picture. Split "build a game" between two subagents and one may build a retro-arcade aesthetic while the other builds a character in a completely different style. Neither can see what the other decided. The agent that merges them inherits an incoherent brief. Passing the original task to both does not fix it, because the decisions that conflicted were made during the work, not stated in the task.
The mitigation is to share the full trace rather than a summary of the task, wherever the receiving agent is continuing the work.
What multi-agent actually buys
Diagram, as text
- 200 documents leads to Parallel summarise
- Parallel summarise leads to Batch into groups of 10
- Batch into groups of 10 leads to Intermediate syntheses
- Intermediate syntheses leads to Single final pass with global view
- 200 documents leads to Structured fields
- Structured fields leads to Deterministic aggregation in code
- Deterministic aggregation in code leads to Single final pass with global view
- Single final pass with global view leads to Report
Three things, none of which is "more intelligence":
Context isolation. A subagent can burn tens of thousands of tokens exploring and return a distilled summary of one or two thousand. That is a ten-to-twenty-fold compression, and it is the strongest honest argument for the pattern. The parent's window stays clean while the work still gets done.
Wall-clock parallelism. Genuinely independent work can run concurrently. Note carefully what this does and does not buy: parallelism reduces latency, never tokens. You pay full price for every branch and simply wait less.
Independent verification. A reviewer that never saw the work being reviewed catches things the author cannot. This is the one benefit that a bigger single agent genuinely cannot replicate.
Topologies
| Topology | Mechanism | Fits | Main failure mode |
|---|---|---|---|
| Orchestrator / worker | Lead decomposes dynamically, dispatches, synthesises | Breadth-first search, unpredictable subtask count | Over-spawning; lead becomes the bottleneck it was meant to relieve |
| Pipeline | Each stage consumes the last stage's output | Fixed, well-understood decomposition | Compounding error; every stage is a lossy boundary |
| Generator–critic | One produces, a separate one evaluates, loop until a gate passes | Clear evaluation criteria | Worthless if the critic is not independent |
| Blackboard | Agents self-select against posted requests | Coordinator cannot model each agent's capability | Nobody owns the task; hard to bound cost |
| Debate | Agents answer, see each other, revise | Narrow — uncorrelated strong models only | Conformity: correct answers get talked out of |
| Hierarchical | Multi-level dispatch and aggregation | Oversight, per-branch budgets, audit | Latency stacks at every level before any work starts |
Debate deserves a specific warning. Evaluations of debate frameworks have repeatedly found them losing to far simpler baselines — plain chain-of-thought, or self-consistency at matched cost — and studies of debate dynamics have measured more correct-to-incorrect flips than the reverse. Models trained to be agreeable are, unsurprisingly, agreeable: a stronger model will abandon a correct answer to align with weaker peers.
Share context with collaborators, withhold it from verifiers
These two rules look contradictory and are both right, because they apply to different roles.
An agent extending the work needs the full trace. Give it a summary and you get the incoherent-merge failure above.
An agent judging the work needs a clean context. A reviewer that has been carried along through every dead end has absorbed the author's assumptions and will reproduce them. Give it the diff and let it rediscover what it needs.
The design question is never "share or isolate". It is "is this agent continuing the work, or checking it?"
Errors compound
A pipeline of independent steps multiplies its success rates rather than averaging them:
| Per-step success | 5 steps | 10 steps | 20 steps |
|---|---|---|---|
| 95% | 77% | 59% | 36% |
| 90% | 59% | 35% | 12% |
| 85% | 44% | 20% | 4% |
A 95%-reliable step is an excellent step, and ten of them in a row fail more often than a coin flip. This is why long agent chains disappoint in ways that no individual component explains.
The escape is not better steps — it is verification that breaks the chain, catching an error before it propagates. Which brings us to the constraint that most multi-agent designs get wrong.
The strongest verifier is not another model at all. A test suite, a type checker, or a compiler gives deterministic, uncorrelated feedback, and it is cheaper than every alternative.
One writer
The rule that production teams converged on: multiple agents may contribute intelligence; only one should take actions.
Parallel writers make conflicting implicit decisions — about interfaces, error handling, naming — and the result compiles while disagreeing at runtime. Read-only subagents are safe precisely because they cannot conflict; they behave more like context-compressing tool calls than collaborators.
Where parallel writes are genuinely required, isolate at the filesystem: separate working directories or branches per agent, with an explicit merge step and automated gates. Isolation prevents file conflicts; it does not prevent semantic conflicts, so decompose along domain boundaries rather than by file, and let tests arbitrate.
Evaluating the system, not the agents
Per-agent evaluation is misleading here by construction: every agent can pass on its own inputs while the composition fails. Failure studies of multi-agent frameworks find a large share of failures in the seams — inter-agent misalignment, information asymmetry, agents disobeying their role — which per-agent tests cannot see.
Grade the end state rather than the path, since agents legitimately find different routes to the same goal, and use checkpoints where you genuinely need to verify intermediate state. Above all, report cost and wall-clock beside quality: a multi-agent evaluation without a token-matched single-agent baseline cannot be interpreted.
Check your understanding
Common mistakes
Reaching for agents when a bigger loop would do. The cheapest experiment is more budget on one agent. Run it first.
Over-decomposition. Each split adds a lossy handoff and another term to the compounding-error product, for no parallelism benefit on small tasks.
Relay agents. An agent that receives and forwards adds latency, tokens and a failure mode while adding no information — the data-processing inequality guarantees it cannot add any.
Unbounded spawning. Without depth and fan-out caps, simple queries can spawn dozens of agents. Cap children, depth and rounds.
Parallel writers. The most consistently condemned pattern in the field, and the one that produces bugs that compile.
Self-critique loops. A model grading its own output rewards what sounds right. That is a single agent with extra steps, not verification.
Copying the task down instead of the trace. The conflicting decisions were made during the work, not stated in the task.
Judging per-agent instead of end-to-end. The seams are where multi-agent systems fail, and per-agent tests are blind to them.
Practice
A team proposes five specialised agents — researcher, planner, coder, reviewer, tester — for a coding task. Evaluate the proposal and say what you would build instead.
Write your own answer first, on paper or in a note. Then compare.
Show a hint
Ask which of the five need to be separate agents rather than separate steps or tools.
Show a strong answer
What the proposal gets right: the five roles are real activities, and separating concerns is sound. Verification by something other than the thing that produced the work is genuinely valuable.
What is wrong with making them five agents:
- Four handoffs, each lossy. The researcher's understanding of an edge case reaches the coder only if the planner thought to write it down.
- Five times the cost, and slower, since the chain is sequential.
- Debugging is far harder. When the output is wrong, the failure is somewhere in five trajectories and four translations.
- Three of the five are not agents at all. The tester runs a test suite — that is a tool, deterministic and cheap. The reviewer is mostly a linter plus a diff check. Making them model calls replaces reliable checks with unreliable ones.
What I would build instead: one agent, better tools, and one genuine second opinion.
- A single agent holding full context throughout, with tools for search, read, edit, and run-tests.
- Verification as tools, not agents — the test suite, the compiler, the linter. Deterministic, fast, and they cannot be talked out of a failure.
- One separate reviewer agent, and only this one, invoked on the final diff. It is worth separating precisely because it should not share the coder's context — a reviewer that saw the reasoning inherits its blind spots. This is the case where withholding context is the point.
The general rule this illustrates: separate agents when you need independence (a check that must not share the original reasoning) or genuine parallelism (independent subtasks with no shared state). Do not separate them to model a workflow — steps in a workflow are steps, and most of them are tools.
You have a task that genuinely parallelises: summarise 200 documents and produce a combined report. Design it, and say where multi-agent helps and where it does not.
Write your own answer first, on paper or in a note. Then compare.
Show a hint
Fan-out is easy. The interesting part is the join.
Show a strong answer
Where parallelism genuinely helps: the 200 summaries. They are independent — no summary depends on another — so this is embarrassingly parallel and the wall-clock saving is close to linear. This is a real case for concurrent workers.
But note what these workers are. Each does one call with one prompt and no loop, no tools and no decisions. They are not agents; they are parallel function calls, and calling them agents obscures how simple the fan-out actually is.
Where it does not help: the join. Combining 200 summaries into a coherent report is inherently sequential and needs global context. Splitting it across agents recreates the handoff problem — each writes a section without seeing the others, and the report reads like five documents stapled together, with contradictions between sections.
How I would build the join:
- Hierarchical reduction. Combine summaries in batches of ten into intermediate syntheses, then combine those. Each step has manageable context and the tree is shallow. Contradictions surface at each level where they can still be reconciled.
- A single final pass over the intermediate syntheses, by one call with the whole picture, to produce the narrative and resolve conflicts.
- Deterministic aggregation for anything countable. Totals, frequencies and date ranges should be computed in code from structured fields the summarisers emit, not written in prose by a model reading 200 documents. This removes a whole class of arithmetic errors.
The design point: parallelise the part with no dependencies, and keep the part that needs a coherent view in one place. Fan-out is a compute decision; the join is a design decision, and the quality of the report is decided almost entirely by the second.
Key takeaways
- Multi-agent is a context and compute allocation strategy, not an intelligence strategy.
- Match token budgets before comparing. Most reported gains shrink or vanish when you do.
- Every handoff is lossy — decomposition has an information cost, not just an engineering cost.
- The three real benefits are context isolation, wall-clock parallelism, and independent verification. Parallelism cuts latency, never tokens.
- Share the full trace with collaborators; give verifiers a clean context. Opposite prescriptions, different roles.
- Success rates multiply across steps — ten 95% steps are a 59% pipeline. Verification is what breaks the exponential.
- Verification only works when errors are uncorrelated: different model families, or a deterministic oracle. Consensus is not correctness.
- Keep one writer. Extra agents may contribute intelligence; concurrent actions produce conflicts that compile.
Further reading
- Qingyun Wu et al., "AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation" (arXiv, 2023).
- Sirui Hong et al., "MetaGPT: Meta Programming for Multi-Agent Collaborative Framework" (ICLR 2024).
- Anthropic, "Building effective agents" (2024) — the argument for simplicity before multi-agent.
- Jeff Dean and Sanjay Ghemawat, "MapReduce" (OSDI 2004) — fan-out and hierarchical reduction, which is what the join above is.
- Melvin Conway, "How Do Committees Invent?" (Datamation, 1968) — communication structure shapes output, for agents as for teams.