Take a capable model and give it a text box, and you get a chat assistant. Give the same model a filesystem, a shell, a test runner, a permission model and a loop, and you get something that can complete a multi-hour engineering task.
The model did not change. The harness did.
A harness is everything surrounding the model that turns raw next-token prediction into a working agent: the system prompt, the tool set, how context is assembled each turn, what the agent is allowed to touch, how results and errors are fed back, and how the whole thing is observed. Harness engineering is the discipline of building that layer well — and in production systems it accounts for as much of the delivered capability as the model choice does.
Why the harness matters as much as the model
The clearest evidence comes from agentic benchmarks. The same published model scores very differently depending on the scaffold running it, and the spread between a naive harness and a well-built one on the same model is routinely larger than the gap between adjacent model generations.
This has an uncomfortable corollary that is worth internalising: when your agent underperforms, the model is usually not the constraint. Upgrading is the expensive, slow lever. Fixing how errors are surfaced, how tools are shaped, and what the agent can see is the cheap, fast one.
The components
Diagram, as text
- Agent proposes action leads to Harness
- Harness leads to Tool with narrow schema
- Tool with narrow schema leads to Sandbox: ephemeral, no network, no credentials
- Sandbox: ephemeral, no network, no credentials leads to Deterministic verification: tests, compiler, validator
- Deterministic verification: tests, compiler, validator leads to Result (pass)
- Deterministic verification: tests, compiler, validator leads to Error as an observation (fail)
- Error as an observation leads to Agent proposes action
| Component | What it owns | The decision that matters |
|---|---|---|
| Instructions | Identity, constraints, output contract | What is a rule versus a default |
| Tool set | The action space | Granularity — how much work one call does |
| Context assembly | What the model sees each turn | What to include, and what to drop |
| Permissions | What the agent may touch | Which actions need a human |
| Execution environment | Where actions happen | Isolation and blast radius |
| Feedback | How results and errors return | Whether errors are actionable |
| Verification | How "done" is checked | Whether the check is independent |
| Observability | What you can reconstruct afterwards | Whether a failure is diagnosable |
Most harness quality lives in the last four rows, which is the opposite of where most attention goes.
Tool granularity is the first design decision
The shape of your tool set determines how many steps a task takes, and step count drives both cost and compounding error.
Tools that are too fine-grained force the model to orchestrate trivia. A task needing twenty calls to read_line, write_line and move_cursor has twenty chances to go wrong and burns context on mechanics. Tools that are too coarse become unpredictable — a single do_the_task tool hides the decisions you wanted the model making explicitly.
The useful heuristic: shape tools around units of work a competent human would name. run_tests, search_codebase, apply_edit. Not keystrokes, and not entire jobs.
Errors are prompts
This is the highest-leverage and most-neglected part of a harness.
When a tool fails, the error message goes straight into the model's context and becomes the basis for its next action. An error is therefore not a log line — it is an instruction, and it should read like one.
Weak: Error: ENOENT
Better: File not found: src/uti1s.ts. Similar files exist:
src/utils.ts, src/util.ts. Check the path and retry.The second version routinely turns a failed run into a recovered one, and it required no model change. Distinguish retryable from terminal explicitly, because a model that cannot tell them apart will either retry forever or give up on a transient failure.
Verification belongs inside the harness
Agents are unreliable narrators of their own success. Left to self-assess, a model will report a task complete because the work looks complete.
The fix is to make the harness check, not the model. Run the tests. Run the type checker. Re-read the file that was supposedly edited. These are cheap, deterministic and completely uncorrelated with the model's own judgement, which is exactly what makes them valuable — a model grading its own output rewards what sounds right.
The strongest harnesses close this loop automatically: after an edit, tests run and failures return as observations, so the agent sees its own mistake without a human intervening.
Isolation and permissions
An agent that can act can act wrongly, so the environment must bound the damage rather than relying on the agent's judgement.
Isolate the workspace. Containers, ephemeral VMs, or per-agent git worktrees give the agent a real environment to work in while keeping mistakes contained and reviewable.
Default to reversible. Prefer actions that can be undone — a branch rather than a push, a staged change rather than a commit, a soft delete rather than a hard one. Reversibility is what lets you grant autonomy without asking permission constantly.
Gate on consequence, not on category. Reading is free. Writing inside the workspace is usually fine. Anything irreversible, externally visible or expensive should stop and ask. Approving every action produces rubber-stamping, which is worse than not asking.
Observability is a harness feature
When an agent does something inexplicable, you need to reconstruct the run: what was in context, which tools were called with which arguments, what came back, and where it diverged.
Log at the boundaries you control — prompts, tool calls, tool results, decisions — because the model's internal reasoning may not be available to you. A harness you cannot replay is a harness you cannot improve, and the failures that matter are exactly the ones that do not reproduce on demand.
Evaluating the harness, not the model
Harness changes need the same discipline as model changes: hold the model fixed and measure the harness.
Keep a task set that exercises real workflows end to end, and measure task success, steps to completion, recovery rate after a failed action, and cost per solved task. Change one thing at a time. A better error message, a merged pair of tools, or an automatic test run after edits will each move these numbers, and without a fixed task set you will have no idea which change helped.
Check your understanding
Common mistakes
Blaming the model for harness failures. Weak error text, missing tools and unverified completion look like model incompetence.
Tools shaped like API endpoints. Your internal service boundaries are not the units of work an agent reasons in.
Exposing every tool you have. Selection accuracy falls as the tool count rises, and every definition costs context on every request.
Errors written for logs. The model is the reader. Write for it.
Trusting the agent's report of success. Verify with something deterministic.
Approval on everything. Fatigue turns review into rubber-stamping.
No replay. Without full traces, you are guessing at causes of failures you cannot reproduce.
Changing several harness elements at once. You will not know which one mattered.
Practice
An agent that edits code succeeds 45% of the time. You may change the harness but not the model. List what you would change, in order of expected impact.
Write your own answer first, on paper or in a note. Then compare.
Show a hint
Look at what the agent can see, what it can check, and how big each step is.
Show a strong answer
In order of expected return:
- Run the tests and feed the result back. If the harness does not already do this, it is the largest single change available: it converts "the model believes this is correct" into "the suite passes or it does not", and gives the loop something concrete to iterate on. Most failures at 45% are edits that look plausible and do not work.
- Make error output usable. A compiler error or stack trace pasted verbatim is far more actionable than "the build failed". Include the file, the line and the surrounding context — the agent cannot open a log it was not given.
- Fix tool granularity. If editing means "replace this whole file", every change risks clobbering unrelated code. A targeted edit tool that replaces a specific range, and fails loudly if the anchor text is not found, removes a class of silent damage.
- Give it the ability to look before editing. Many failures are edits made without reading enough surrounding code. Cheap search and read tools, and a prompt that expects them to be used first, address that.
- Add a lint and format pass automatically, so trivial issues never consume loop iterations.
- Bound and report. Cap iterations, and when the cap is hit, surface the trajectory rather than a bare failure — so the 55% produces diagnosis rather than silence.
What I would not start with: more elaborate prompting about being careful. It is the cheapest thing to try and the least effective, because the constraint is usually information the agent never had, not diligence it failed to apply.
Design the isolation for a harness that lets an agent execute code it has written. State what it can reach and what happens when it misbehaves.
Write your own answer first, on paper or in a note. Then compare.
Show a hint
Assume the code is hostile, because content the agent read may have instructed it.
Show a strong answer
Design on the assumption that the code is hostile. This is not paranoia: the agent may have read a file, a web page or a dependency containing instructions, and prompt injection turns "the agent's code" into "the attacker's code".
What it runs in: an ephemeral container or microVM, created per task and destroyed after, with no persistence between runs. A fresh environment means contamination cannot accumulate.
What it can reach:
- A working directory it owns, and nothing else on the filesystem. No mounted home directory, no access to the host's source tree, no credentials files.
- No network by default. If dependencies are needed, allow only a package mirror, through a proxy, on an allow-list. This is the control that prevents exfiltration — the most likely goal of injected code.
- No cloud credentials, no metadata endpoint, no internal services. The metadata endpoint in particular must be blocked explicitly; it is the standard escalation path.
Resource limits: CPU, memory, wall-clock and disk, all enforced by the runtime rather than by cooperation. An infinite loop or a fork bomb must be stopped by the sandbox, not by the agent noticing.
What happens when it misbehaves:
- The container is destroyed, and that is the entire remediation — nothing outside it changed.
- The event is logged with the code that ran and the trajectory that produced it, because a sandbox violation is a signal worth investigating rather than a routine failure.
- The agent receives the outcome as an observation — "process exceeded memory limit and was terminated" — so it can adapt rather than being cut off silently.
The one thing that undermines all of it: giving the sandbox a credential "just for this integration". Every such exception makes the isolation decorative, and the exceptions accumulate quietly.
Key takeaways
- The harness is everything around the model — and it determines delivered capability as much as the model does.
- When an agent underperforms, check the harness first; it is the cheaper and faster lever.
- Shape tools around units of work a human would name, and treat descriptions as executable interface.
- Errors are prompts. Make them actionable and mark retryable versus terminal.
- Verify with something deterministic — tests, type checkers, re-reads — because self-assessment rewards what sounds right.
- Isolate the workspace, prefer reversible actions, and gate on consequence rather than asking about everything.
- Everything entering context is untrusted input; containment is structural, not textual.
- Evaluate the harness with the model held fixed, measuring success, steps, recovery and cost per solved task.
Further reading
- Anthropic, "Building effective agents" (2024) — harness design over model selection.
- Carlos Jimenez et al., "SWE-bench: Can Language Models Resolve Real-World GitHub Issues?" (ICLR 2024) — where harness quality dominates measured results.
- John Yang et al., "SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering" (NeurIPS 2024) — the interface as the variable.
- OWASP, Top 10 for Large Language Model Applications (2023) — insecure plugin design and excessive agency.
- Google, "gVisor: Container Runtime Sandbox" (technical documentation) — isolation for untrusted code.