Key takeaways
- To learn how to build an agent harness, treat the harness as everything around the model.
- Start with the failure you can least afford, then bound the agent's authority.
- Preserve state, verify actions independently, and define recovery before the agent gets risky tools.
What is an agent harness?
An agent harness is the operating system around a model. In the coding-agent context, a useful shorthand is Agent = Model + Harness. The model supplies capability. The harness supplies the conditions under which that capability can act.
That distinction becomes concrete as soon as an agent gets tools. An agent can plan its own steps and call tools that read files, run commands, or edit code. It can also direct its process and tool use with little human input. The harness has to govern those actions, retain the right context, and decide what happens after each result.
This is the core concern of harness engineering. Model capability and execution reliability are different properties. A capable model can still operate inside a weak execution system. When an agent fails, the investigation can cover the task definition, available context, environment configuration, verification, and state retained between sessions. Those are harness questions, not just model questions.
Treat the equation as an engineering boundary. Inspect the model's plan separately from the execution system. The system needs to retain the state required by the task and determine whether an action succeeded.
This separation also makes a working demo easier to examine. A convincing response shows what the model can produce in one run. Execution reliability asks a different question: what happens when that run uses tools, carries state into another session, or reaches a result that needs verification? Keeping those properties distinct gives the harness a job that can be specified and tested.
So the first design choice is not which framework has the longest feature list. We frame harness selection around the failure you can least afford. That failure gives you a concrete place to begin setting authority, state, checks, and recovery.
How to build an agent harness from failure boundaries
If you want to build an agent harness from scratch, write down the unacceptable failure before you write the loop. Unchecked errors in an autonomous workflow can compound. A bad decision can change the context for later steps and obscure the original cause. Starting from that exposure forces the control layer to answer a real operating problem.
The Failure-Backward Harness Canvas records ten decisions in five connected areas. Use it for one agent task, not an imagined platform that does everything.
| Design area | Questions | Recorded decision | Example |
|---|---|---|---|
| Failure and side effect | What failure can you least afford? What side effect follows? | Name the failure and the affected action or state. | A bad decision changes later context and hides its original cause. |
| Authority and behavioral limit | Which tool may act? What hard limit applies? | Set permission tool by tool, plus a call, write, record, sequence, or cooldown limit. | Require approval before an irreversible or costly action. Cap tool calls per task. |
| Durable state and checkpoint | What must survive interruption? After which meaningful step? | Record the state to retain and the checkpoint boundary. | Save complete agent state so a restart can resume from the latest checkpoint. |
| Uncertain outcome and recovery | What happens when a persisted call has no result? | Mark the external outcome unknown and choose a recovery path that does not retry automatically. | A call may have completed externally even though no result was recorded. |
| Verification, observability, and regression evaluation | What independent check gates the next step? What signal reveals trouble? Which failure becomes an evaluation? | Record the check, monitoring signal, and failure-derived regression case. | Gate the loop on an independent check, group interruptions by severity, and turn the failure into an evaluation case. |
The authority row deserves precision. Agent safety requires permission decisions tool by tool and boundary by boundary before risky actions ship. Behavioral limits can include maximum tool calls per task, maximum writes per hour, maximum records returned, disallowed tool sequences, and cooldowns after sensitive actions. Approval gates place a checkpoint before side effects occur, which reduces the impact of hallucinations and mistaken tool calls.
What are the key controls around the model?
The controls around a model do three different jobs: guide behavior before action, sense what happened after action, and compute an independent result.
Feedforward guides anticipate agent behavior and steer it before the agent acts. In a coding harness, that can be an instruction or guide that defines the expected behavior before a tool call. It gives the loop direction while a choice is still open.
Feedback sensors observe the agent after it acts and help it self-correct. A sensor can expose the result of an action to the loop, so the next iteration responds to what happened instead of relying only on the prior plan.
Computational controls include tests, linters, type checkers, and structural analysis. These controls are deterministic and fast. A test can check the result of a code change, while a type checker can reject an invalid change before the loop continues.
These controls also create different places to intervene. A feedforward guide changes what the agent sees before choosing an action. A feedback sensor changes what it can observe after the action. A computational control produces a separate result that the loop can use as a gate. In a plan-act-verify loop, the agent states its intended step, uses the tool, and passes that independent check before another iteration begins.
The useful pattern is not a large prompt that tries to predict every problem. Put guidance before the action, capture feedback after it, and use a computational check wherever the output has a testable structure. The agent harness glossary provides a compact reference for the wider control layer these parts form.
Where should human approval enter the loop?
Human approval belongs where the consequence justifies the interruption. Read-only intelligence work can run autonomously. Write actions and external communications should remain supervised by default until they have been proven safe.
That gives a practical boundary for early harnesses. Read-only summaries, retrieval, classification, and drafts can stay on the autonomous side because they are intelligence work. A write or external communication crosses into supervised action. Approval is especially appropriate when the next action is irreversible, costly, regulated, or has a high blast radius.
The checkpoint decision should also account for four operating conditions: high irreversibility, low model confidence, external visibility, and missing context. These conditions put review around consequence and uncertainty instead of forcing a person to approve every routine step. A resumable workflow can pause at that checkpoint and continue later with its complete state preserved.
Apply both parts of the policy. First classify the proposed action by authority: read-only intelligence, a write, or an external communication. Then check whether the moment has high irreversibility, low confidence, external visibility, or a context gap. Read-only work can still reach a checkpoint when confidence is low or context is missing. A write can remain supervised even when the model reports high confidence.
This makes the approval rule legible to the operator. The checkpoint is tied to the proposed action and its consequence, not scattered across every internal thought or routine tool response. The agent can keep doing bounded intelligence work while a consequential action waits with its state preserved.
Review policy should not remain static. Track the override rate at each checkpoint. Frequent overrides reveal where human judgment is still useful. A low override rate can reveal a checkpoint that may be relaxed. The goal is measured oversight at the points that continue to earn it, with authority widening only after the operating record supports that change.
Verification must inspect outputs and trajectories
Do not accept the agent's own report as proof that the work succeeded. Verification should use an independent check. Depending on the output, that check can be a test, type check, linter, schema validation, or a second model grading the result.
Output quality and execution behavior are separate evaluation targets. An LLM judge can score an agent response against flexible criteria. Trajectory scoring can inspect tool use and the sequence of actions with scoring functions. A response may satisfy the final rubric while its trajectory exposes a tool choice or action sequence that the harness should catch.
Record the two targets separately. The response score applies criteria to what the agent returned. The trajectory score applies scoring functions to the tools and action sequence used along the way. That separation lets an evaluation retain a useful final answer while still identifying execution behavior that should fail a harness gate.
The independent check can also sit directly inside the loop. The agent proposes and performs an action, but the test, schema validator, type checker, linter, or second model produces the verification result. The loop continues only after that check, rather than accepting a natural-language claim that the action worked.
This is why agent evaluation belongs inside the loop design. The result answers whether the task ended well. The trajectory shows how it got there. Both can become explicit gates and regression signals.
Durable execution has a side-effect boundary
Durable execution keeps agent code moving through infrastructure crashes, API timeouts, rate limits, and deployment restarts. It can preserve the workflow's progress across interruptions that would otherwise end an in-memory loop.
That durability also supports waiting. A resumable workflow can pause for human approval or an external event, then resume with complete state preserved. This makes a checkpoint an execution state, not a request held together by a process that must stay alive.
Design both interruption paths explicitly. Infrastructure crashes, API timeouts, rate limits, and deployment restarts test whether execution can recover. Human approval and external events test whether it can wait. Durable execution is designed to cover the first group, while resumable workflows preserve complete state across the second.
But durable intent does not guarantee exactly-once external effects. A workflow may know that it intended to call a side-effecting tool without being able to prove whether the external action completed. When the provider supports one, pass an idempotency key to the side-effecting tool. That gives the external system a way to recognize a replay of the same intended action.
Keep this boundary visible in the recovery policy. Checkpointing protects the agent's state. Replay safety for an external effect depends on the tool and provider. When no recorded result can settle the external outcome, preserve the unknown state instead of treating a restart as permission to repeat the action.
Monitoring feeds the next evaluation cycle
Monitoring should expose whether the workflow is progressing, failing, or waiting. Useful workflow measures include:
- execution duration;
- activity failure rates;
- retry attempts and backoff; and
- pending workflow counts.
Those signals identify the cases worth turning into evaluations. Build evaluations from realistic, failure-derived tasks, then give each task unambiguous success criteria. Combine grader types thoughtfully, keep the problems difficult enough to reveal weaknesses, improve the signal through iteration, and review transcripts.
Give each metric an operator question. Which executions are taking the longest? Which activities fail most often? Where are retries and backoff accumulating? How much work is still pending? The answers describe execution behavior that a final response alone cannot show.
Then preserve the relevant failure as a task. Define success without ambiguity and select the graders needed for that criterion. A difficult case is useful because it keeps producing a meaningful signal as the harness changes. Iteration improves that signal, while transcript review checks the action sequence behind the score.
This creates a working relationship between production and evaluation. Monitoring exposes a concrete failure or delay. The evaluation preserves that case and tests whether a later harness change regresses. Transcript review then shows whether the score reflects the behavior you intended to measure.
What do agent harnesses look like in practice?
Agent harnesses appear in several implementation shapes. Each one puts the control layer at a different boundary.
One shape is an in-process programmable loop. The Agent SDK supplies tools, an agent loop, and context management through Python and TypeScript interfaces. A developer can run that loop inside their own Python or TypeScript process. This shape puts the loop beside application code while leaving the developer responsible for the surrounding authority, verification, state, and recovery decisions.
A second shape is a control layer around a coding agent. We treat BMAD and Spec Kit as agent harnesses in this sense. They wrap coding agents with control instead of standing in for the underlying model. This is a useful example of the Model + Harness equation becoming an implementation boundary.
A third shape adds durable execution to a multi-agent system. Swarm can be augmented with durable execution to support reliable, scalable multi-agent systems. Here, the harness concern includes preserving execution across the work performed by multiple agents.
The programmable loop is embedded in a developer process. The coding control layer wraps an existing coding agent. The durable multi-agent shape adds execution continuity around coordinated agents. These are concrete examples of harnesses, even though their boundaries and execution models differ.
Harness engineering can reach the whole software system
Harness engineering is not limited to wrapping one tool call. In one five-month internal beta experiment, Codex produced a software product with zero manually written lines of code. It wrote the application logic, tests, continuous-integration configuration, documentation, observability, and internal tooling.
That experience shows the scope a harness may need to govern when agent work reaches beyond a single code change. The relevant surface can include the application, its tests, the delivery configuration, the documentation, the signals used to observe it, and the tools used to support the work. The experiment is one organization's experience, not a universal performance claim.
It also makes the phrase "harness engineering" concrete at system scale. The agent's work crossed source code, verification, delivery, documentation, observability, and supporting tools during the same experiment. Each category became part of the software-producing environment around the model.
Start smaller. Put one real agent task into the Failure-Backward Harness Canvas and make every authority, state, check, and recovery field concrete. Then read our note on building agentic AI systems to place that task inside a wider system design.