How it works
Outer-loop optimization operates across completed runs. It treats the harness itself as the object being improved. The input is not merely a model response, but evidence from one or more system candidates: execution traces, task outcomes, evaluator scores, operator corrections, cost records, and failure classifications.
A useful outer loop has five steps:
- Capture a baseline. Record the candidate’s version, environment, inputs, permissions, and evaluation results.
- Identify a bounded adaptation surface. State which prompts, policies, routing rules, tool descriptions, or code paths may change.
- Propose a candidate. Generate a concrete, reviewable change rather than a general recommendation.
- Evaluate the candidate. Run representative tasks, negative controls, and regression checks against the baseline.
- Promote or reject it. Adopt the candidate only when it meets the promotion policy; otherwise retain the previous version.
The loop should produce an artifact that can be inspected: a patch, configuration diff, evaluation report, and decision record. That separation matters. The optimizer may recommend a change, but a deterministic control path decides whether the change reaches production. The current production harness remains stable until a candidate clears that boundary.
Why it matters in an agent harness
Agent behavior depends on more than model weights. Prompts, context assembly, tools, permissions, retry rules, memory, evaluators, and stop conditions all shape what the system does. Outer-loop optimization gives teams a way to improve those mechanisms from operational evidence without allowing each live run to rewrite its own controls.
That boundary supports reversibility. Every promoted candidate can carry a version, evaluation record, and known predecessor. If a change increases false completion, expands tool use, or violates a behavioral invariant, operators can identify the responsible version and roll it back. Without that lineage, adaptation becomes configuration drift with a positive label.
It also improves observability. A score increase is not enough. The loop should preserve the traces needed to explain what changed: whether the candidate used fewer retries, selected a different tool, stopped earlier, or merely learned to satisfy a weak evaluator. This is especially important when the proposal mechanism and the evaluator are both model-based. They can share blind spots and produce apparent progress without better real-world behavior.
Permissions should remain outside the optimizer’s unilateral authority. A candidate may propose a broader scope because it makes a benchmark easier, but benchmark performance does not justify additional access. Changes to credentials, write scopes, approval gates, or execution limits need a separate authorization policy.
Outer loops also impose operating cost. Candidate generation, replay, evaluation, and review consume time and compute. The practical goal is therefore not continuous mutation. It is selective improvement: change the smallest relevant surface, test it against meaningful cases, and retain the evidence required to reverse the decision.
Outer-Loop Optimization vs Evaluator-Optimizer
Both patterns use feedback, but they act at different times and on different objects. Confusing them can accidentally give a task-level agent authority to alter the harness that constrains it.
| Dimension | Outer-loop optimization | Evaluator-optimizer |
|---|---|---|
| Object changed | Harness code or configuration | A task output or intermediate artifact |
| Time horizon | Across completed runs or releases | Within one task execution |
| State | Versioned candidate history | Current task state |
| Typical evidence | Traces, regressions, aggregate outcomes | Critique of the current output |
| Promotion boundary | Explicit evaluation and deployment decision | Loop stop condition or acceptance check |
| Main risk | Harness drift or evaluator gaming | Unbounded iteration or local overfitting |
An evaluator-optimizer loop might revise a generated implementation until tests pass. An outer loop might examine many such executions and propose a different retry policy or tool-selection rule for the next harness version. The first optimizes work inside the boundary. The second proposes moving the boundary itself.
The Rifty take
We treat outer-loop optimization as controlled software change, not automatic self-improvement. We optimize for attributable proposals, narrow adaptation surfaces, and reversible promotion decisions, accepting slower change in exchange for knowing which mechanism changed and why. The optimizer may propose; it should not silently acquire the authority to redefine its own constraints.
Implementation checks
- Version the baseline, candidate, evaluator, and task set together.
- Define the adaptation surface before generating a proposal.
- Require a concrete diff rather than an unstructured recommendation.
- Keep permission expansion behind a separate approval boundary.
- Test behavioral invariants and negative controls, not only average scores.
- Inspect execution traces for evaluator gaming and shifted failure modes.
- Record rejected candidates as well as promoted ones.
- Preserve a tested rollback path to the previous harness version.
- Re-evaluate after promotion using live operational evidence.
- Stop the loop when evidence is insufficient, contradictory, or outside the evaluator’s competence.
Frequently asked questions
What should an outer-loop optimizer be allowed to change?
An outer-loop optimizer should change only a declared adaptation surface, such as selected prompts, routing rules, tool descriptions, retry policies, or configuration values. Permission scopes, credentials, approval requirements, and other security boundaries should require separate authorization, even when changing them would improve an evaluation score.
How often should outer-loop optimization run?
It should run when enough comparable evidence exists to justify evaluating a change, not merely because another task completed. The useful cadence depends on task volume, evaluation cost, and risk. High-impact harnesses generally benefit from larger evidence windows and more deliberate promotion than low-risk experimental systems.
Can an outer-loop optimizer deploy its own changes?
It can technically be given that authority, but the safer default is separation between proposal and promotion. A deterministic policy or accountable operator should verify evaluations, invariants, permissions, and rollback readiness. Automatic promotion is reasonable only within a tightly bounded, low-impact surface with reliable regression detection.
How do we detect evaluator gaming in the outer loop?
Compare evaluator scores with execution traces, side effects, negative controls, and independent outcome checks. Rotate or version evaluation cases, and inspect whether gains come from the intended mechanism. A candidate that improves a score while weakening task completion, permissions, or failure reporting should not be promoted.
What makes an outer-loop change reversible?
A reversible change has a versioned predecessor, a concrete diff, compatible state handling, and a tested rollback procedure. Reverting code alone may be insufficient when the candidate changed durable memory or workflow state. The promotion record should identify those effects and the compensating actions required.