An agent is a loop. It gathers context, takes an action, observes the result, and decides whether to continue. Everything else — tools, memory, prompts — exists to serve that cycle.
The loop itself is where agents actually fail. Not by choosing the wrong tool once, but by never stopping, by stopping too early, by repeating the same failed action eleven times, or by filling their own context with noise until they can no longer see the task. Loop engineering is the design of the control flow that prevents those outcomes.
It is distinct from its neighbours. Context engineering asks what goes in the window. Harness engineering asks what surrounds the model. Loop engineering asks when to iterate, when to stop, and what carries forward.
The shape of the loop
Most production loops reduce to the same four steps:
while not done:
context = assemble(task, history, observations)
action = model(context)
result = execute(action)
done = check(result)The named patterns are variations on where the effort goes. Reason-then-act interleaves deliberation with each action. Plan-then-execute front-loads a plan and follows it. Reflect-style loops add a critique pass before continuing.
The pattern matters less than four decisions that every loop must make regardless: how it terminates, what it spends, what it carries forward, and what it does when something fails.
Termination is the hard part
Diagram, as text
- Start leads to Model proposes action
- Model proposes action leads to Execute tool
- Execute tool leads to Progress since last iteration?
- Progress since last iteration? leads to Abort early, report trajectory (no, repeated action)
- Progress since last iteration? leads to Model says done? (yes)
- Model says done? leads to Verification passes? (yes)
- Verification passes? leads to Success (yes)
- Verification passes? leads to Model proposes action (no)
- Model says done? leads to Budget, iterations, clock left? (no)
- Budget, iterations, clock left? leads to Model proposes action (yes)
- Budget, iterations, clock left? leads to Stop, return partial work (no)
The naive loop exits when the model says it is finished. This fails in both directions, and both failures are common.
It stops too early. The model reports success because the work looks complete. Nothing verified it.
It never stops. The model keeps finding one more thing to check, or oscillates between two states, each time genuinely believing progress is being made.
A robust loop uses several independent exit conditions:
| Condition | Trigger | Outcome |
|---|---|---|
| Verified success | An external check passes | Success |
| Explicit signal | The model calls a done tool with a result | Success, if verified |
| Budget exhausted | Tokens, wall-clock, iterations or cost hit a cap | Partial, reported honestly |
| No progress | N iterations with no change in state | Stop and escalate |
| Repetition | The same action repeats with the same result | Stop and escalate |
The load-bearing insight is that the model should not be the sole judge of its own completion. Where a deterministic check exists — tests pass, the file parses, the record exists — that check is the exit condition and the model's opinion is advisory.
Budgets, and degrading honestly
Every loop needs hard ceilings: maximum iterations, maximum tokens, maximum wall-clock, maximum spend. Uncapped loops are the standard way to turn a small bug into a large invoice.
What matters more than having budgets is what happens when one is hit. The bad version stops mid-action and returns whatever is in the buffer, leaving the work in an unknown state. The good version reserves budget for a clean exit: stop starting new work, finish or roll back what is in flight, summarise what was done and what remains, and return that.
An agent that says "I completed the first three of five steps; here is what is left" is useful. An agent that stops mid-edit is a mess someone has to clean up.
Context is the resource the loop consumes
Every iteration appends: the action, the result, the reasoning. A loop that runs long enough will exhaust its own context window, and quality degrades well before it errors.
Compact deliberately. When history approaches a threshold, summarise older iterations into a compact record of what was learned and what was tried. Do this on a threshold, not at the point of failure.
Decide what must survive verbatim. Summarisation is lossy, and the losses are not random — prose summaries drop specifics first. The original task, hard constraints, decisions already made and exact identifiers belong in structured state carried forward untouched, not in the summary.
Keep failed attempts, briefly. "Tried X, failed because Y" is one of the highest-value lines in the context. Discard it and the loop will try X again.
Delegate to isolate. Where a subtask needs extensive exploration, run it in a subagent with its own window and return only the conclusion. The parent spends a few hundred tokens on something that cost tens of thousands, and its own context stays clean.
Errors as observations
Inside a loop, a failure is not an exception to be thrown — it is an observation to be fed back so the next iteration can respond.
Three rules keep this from degenerating:
Distinguish retryable from terminal. A timeout deserves a retry. A malformed request does not; retrying it verbatim wastes an iteration.
Never retry identically. If the model repeats a failing call unchanged, the loop is stuck. Require the action to differ, or escalate.
Cap retries per action, not just globally. Otherwise one broken tool consumes the entire budget while the rest of the task waits.
Parallelism inside the loop
Where an iteration produces several independent pieces of work, they can run concurrently. Two cautions.
Parallelise reads freely; serialise writes. Concurrent reads are safe. Concurrent writes to shared state reintroduce every race condition you know about, with a non-deterministic actor driving them.
Prefer pipelining to barriers. Waiting for every branch before starting the next stage means paying the slowest branch's latency at every stage. Unless a stage genuinely needs all prior results together — deduplicating across them, for instance — let each item flow through independently.
Durability and interruption
Long-running loops need to survive interruption, whether from a crash, a deploy, or a human stepping in.
That means checkpointing loop state — iteration count, budget consumed, what has been done — durably enough to resume, and making actions idempotent so a replayed step does not duplicate its effect. It also means letting a human interrupt and redirect mid-loop, which requires the same checkpointing. An agent that cannot be steered without being restarted will be restarted, losing all its accumulated context.
Check your understanding
Common mistakes
Letting the model decide when it is done. Where a deterministic check exists, that check is the exit condition.
No no-progress detection. The most expensive failure mode, because it looks like work.
Truncating at the budget instead of exiting cleanly. Reserve budget to finish, summarise and report.
Appending forever. Compact on a threshold, before quality degrades.
Summarising away the task or its constraints. Durable facts belong in structured state, not prose.
Discarding failed attempts. The loop repeats them.
Retrying identical actions. If the action has not changed, the result will not either.
Fixed iteration counts for unknown-size work. Loop until dry instead.
Loops that cannot be resumed or steered. They get restarted, and all context is lost.
Practice
Design the termination conditions for an agent loop, covering success, failure, and everything in between. Say what the caller receives in each case.
Write your own answer first, on paper or in a note. Then compare.
Show a hint
There are more than two ways for a loop to end, and each needs a distinguishable outcome.
Show a strong answer
Six conditions, each with a distinct outcome:
- Success. The model signals completion and a verification check passes. Do not accept the model's word alone where a check exists — "I have fixed it" with a failing test suite is not success. Caller receives: the result plus the verification evidence.
- Verified failure. The agent concludes the task cannot be done — the record does not exist, the permission is absent. Caller receives: a clear negative result with the reason. This is a successful outcome operationally, and counting it as failure distorts every metric.
- Iteration cap reached. Caller receives: partial progress, the trajectory, and an explicit "did not complete within N steps". Never a bare failure — the partial work is often most of the value.
- Budget exhausted (tokens or money). Same shape as above, with the budget named, because this one is a configuration problem rather than a capability problem and the caller should be able to tell the difference.
- Wall-clock timeout. Distinct from the above because it usually indicates a slow dependency rather than a long task, and it should be reported that way.
- No progress detected. Same action with the same arguments N times, or state unchanged across several iterations. Caller receives: an early abort with the repeated action highlighted — the cheapest and most diagnosable stop available.
What every outcome must include: the full trajectory. An agent that fails without showing what it tried is undebuggable, and the trajectory is also the raw material for the evaluation set.
The design point: a loop with two outcomes — worked or did not — hides the difference between "impossible", "too slow", "too expensive" and "stuck", which are four different engineering problems with four different fixes.
Your agent's average task uses 40,000 tokens but the 95th percentile uses 400,000. Investigate and propose controls.
Write your own answer first, on paper or in a note. Then compare.
Show a hint
A tenfold tail is not variance in task difficulty. Something is looping.
Show a strong answer
A tenfold spread is not difficulty; it is a behaviour. Difficulty produces a smooth distribution, and this is a tail.
What I would look at, from the trajectories in the top 5%:
- Repeated identical actions. The clearest signature of an oscillation. Count distinct actions against total actions — a ratio far below 1 is a loop.
- Context growth per iteration. If each tool result is appended in full, context grows superlinearly: iteration 20 re-sends everything from iterations 1–19. Token spend then grows quadratically with loop length, which is usually the actual mechanism behind a tenfold tail rather than ten times as many steps.
- Large tool outputs. One tool returning a 50,000-token file dump, appended and then re-sent every subsequent iteration, single-handedly produces this distribution.
- Task clustering. Are the expensive runs a particular kind of task? If so, that class needs a different approach rather than a bigger budget.
Controls I would add:
- Truncate and summarise tool outputs before they enter the context, with the full output retrievable by reference. This alone usually removes most of the tail.
- Cap context growth explicitly, summarising older iterations once past a threshold, so cost grows linearly rather than quadratically.
- Detect repetition and abort early, rather than letting the budget absorb it.
- Set a per-task budget derived from the median, not the maximum — say 4× the median — so a pathological run is stopped at a known cost rather than discovered on the bill.
The metric to publish: cost per successful task, split by percentile. The mean conceals exactly this problem, and the tail is where the money goes.
Key takeaways
- An agent is a loop; termination, budget, carry-forward and failure handling are the four decisions it must make.
- The model should not be the sole judge of completion — prefer an external, deterministic check.
- Detect no progress by state, not sentiment. A stuck loop looks healthy right up to the budget cap.
- Give every loop hard ceilings, and reserve enough budget to exit cleanly and report honestly.
- Compact on a threshold, keep constraints and identifiers in structured state, and keep failed attempts.
- Subagents isolate context: spend a fresh window on a subtask, return only the conclusion.
- Feed errors back as observations, distinguish retryable from terminal, and never retry an identical action.
- For unknown-size work, loop until dry rather than to a fixed count.
- Checkpoint and make actions idempotent so a loop can be interrupted, steered and resumed.
Further reading
- Shunyu Yao et al., "ReAct: Synergizing Reasoning and Acting in Language Models" (ICLR 2023).
- Noah Shinn et al., "Reflexion: Language Agents with Verbal Reinforcement Learning" (NeurIPS 2023).
- Charles Packer et al., "MemGPT: Towards LLMs as Operating Systems" (arXiv, 2023) — bounding context growth in long loops.
- Michael Nygard, Release It!, second edition (Pragmatic Bookshelf, 2018) — bounded resources and fail-fast.
- Google SRE, Site Reliability Engineering (O'Reilly, 2016), chapter 22 — retries as an amplifier, which applies directly to loops.