How it works
False completion begins when a harness treats an agent’s statement of success as proof that the requested work happened. The model may say a file was changed, a message was sent, or a deployment succeeded. That report describes an intended outcome. It does not establish the resulting state.
A reliable completion path separates execution from acceptance:
- Define a completion predicate before execution. Specify the state that must exist, not merely the action the agent should attempt.
- Run the operation and capture its identifiers, outputs, and errors.
- Read the authoritative system after the operation. A tool response alone may be insufficient when work is asynchronous or crosses a service boundary.
- Evaluate the observed state against the predicate.
- Record success, failure, or an explicitly unknown outcome with supporting evidence.
False completion appears when one or more of these steps collapse into a language judgment. Common examples include accepting a zero exit code without checking the artifact, trusting a browser click without confirming the server-side change, or marking a delegated task complete because a subagent returned a polished summary.
The key mechanism is evidence-bound state transition. Completion should be emitted by the harness only after a verifier observes the required postcondition. The agent can propose that it is done. The harness decides whether the run has earned that state.
Why it matters in an agent harness
A false completion is often more damaging than a visible failure. A failure keeps uncertainty in view. A false success hides it and invites dependent work to continue. An orchestrator may release a lock, delete temporary state, notify a person, launch the next task, or make an irreversible change based on a result that never existed.
The problem becomes sharper in multi-step agent loops. Suppose an agent is asked to update a configuration, run validation, and publish the result. If the update silently fails but the run is marked complete, later validation may inspect an old artifact and publication may distribute it. The initial error has become a provenance problem: the harness can no longer explain which state was actually evaluated.
Completion therefore belongs in the control plane. The harness should own:
- the completion predicate and required evidence;
- the source of truth used for verification;
- the distinction between attempted, accepted, failed, and unknown states;
- the policy for retries, compensation, and operator escalation;
- the durable record connecting a claim to an observed result.
This boundary improves observability because a trace can show both the attempted action and the verification that followed. It improves reversibility because recovery starts from a known state. It also limits permissions: an agent does not need authority to declare its own side effects valid simply because it had authority to attempt them.
Verification should be proportionate to impact. A low-risk formatting task may need a file diff and parser check. A consequential external write may require a subsequent read from the authoritative service, identity matching, and confirmation that the final value equals the proposal. Stronger verification costs time and tool calls, but accepting cheap, self-reported success transfers that cost into debugging and recovery.
The design trap is to ask the same model that performed the task whether it completed the task. Reflection can catch obvious omissions, but it is not independent evidence. When the outcome can be checked deterministically, the harness should prefer that check. When it cannot, the state should remain qualified rather than being promoted to success.
False completion vs unknown outcome
These states can look similar, but they require different operating decisions.
| State | What the harness knows | Correct response |
|---|---|---|
| False completion | Success was reported, but required evidence is absent or contradicts the report. | Reject the completion claim, preserve evidence, and retry or recover according to policy. |
| Unknown outcome | The operation may have succeeded, but observation is unavailable or inconclusive. | Avoid blind repetition, reconcile with the authoritative system, and escalate if ambiguity remains. |
The distinction matters for non-idempotent actions. Retrying a disproven write may be safe. Retrying an unknown payment, publication, or message can duplicate the side effect. A harness should never translate uncertainty into either success or failure merely to keep the workflow moving.
The Rifty take
We treat completion as a verified state transition, not an agent utterance. We accept the latency and implementation cost of explicit postcondition checks where a false success would propagate or trigger irreversible work. If the harness cannot establish the outcome, it should preserve that uncertainty instead of manufacturing closure.
Common failure modes
- Using the agent’s final message as the workflow completion signal.
- Checking that a tool call returned without checking the resulting state.
- Verifying against cached, local, or non-authoritative data.
- Defining success as “no error was observed” rather than a concrete postcondition.
- Letting one successful subtask mask failures across the rest of a fan-out.
- Deleting checkpoints or releasing locks before verification completes.
- Treating asynchronous acceptance as completed processing.
- Retrying an unknown non-idempotent action as though it had definitely failed.
- Recording evidence without binding it to the exact request, target, and execution.
- Allowing verifier drift, where the check no longer matches the run contract.
Frequently asked questions
How should a harness detect false completion?
Define a machine-checkable postcondition and test it against the authoritative system after execution. The check should identify the exact target and expected state, then retain the observation with the run record. An agent’s summary can guide verification, but it should not substitute for evidence.
Is a successful tool response enough to mark a task complete?
Usually not when the operation has an external or asynchronous side effect. A successful response may confirm only that a request was accepted. Completion requires evidence that the intended state now exists, such as a read-back, a validated artifact, or a terminal job status.
What should happen when the result cannot be verified?
Record the outcome as unknown and preserve the request, identifiers, responses, and last observed state. Do not convert uncertainty into success, failure, or an automatic retry. Reconcile with the authoritative system first, especially when repeating the operation could create a duplicate side effect.
Can an LLM judge whether another agent completed a task?
An LLM can assess semantic quality or notice missing work, but it cannot prove an external state change from prose alone. Use model-based evaluation for judgment-heavy criteria and deterministic checks for observable postconditions. High-impact completion may require both, with the harness combining their distinct evidence.