How it works
Contract drift begins when two components share an assumption that is not updated as a single, enforced unit. One side may change a field name, make an optional value effectively required, reinterpret a status, register a different tool signature, or alter what success means. The other side continues operating against the earlier contract.
In an agent harness, the disagreement can span more than a request and response schema. A contract may also cover authority, state transitions, retry behavior, side effects, error classes, and evidence required for completion. Drift occurs when any participant changes one of those expectations without coordinated validation.
A typical sequence is:
- A producer emits an output accepted by its local schema.
- A consumer parses that output but applies older semantics.
- The workflow continues because no boundary check detects the disagreement.
- A later stage fails, repeats an action, or records an invalid success state.
The dangerous cases are not clean parse failures. They are plausible-looking interactions in which both components believe the contract was satisfied. Preventing drift therefore requires versioned contracts, compatibility checks, explicit behavioral invariants, and tests that exercise the complete boundary rather than either component in isolation.
Why it matters in an agent harness
An agent harness coordinates components with different release cycles and failure characteristics: models, prompts, tools, policies, state stores, verifiers, and external services. Contract drift weakens control because the harness can no longer rely on a shared interpretation of an operation.
Consider a tool that once returned completed only after confirming a remote side effect. If a later implementation returns completed after merely accepting the request, an unchanged orchestrator may advance the workflow too early. The JSON remains valid, but the operational meaning has changed. A retry could then duplicate the side effect, while an audit trail incorrectly shows that the first attempt was verified.
Drift also damages reversibility. A compensating action is safe only when it understands the exact state produced by the original action. If identifiers, ownership rules, or completion semantics have shifted, rollback may target the wrong resource or run against an unknown outcome. The same problem appears at permission boundaries: a renamed capability or broadened scope can grant an agent more authority than the policy author intended.
Observability must expose the active contract, not just the payload. Traces should make it possible to determine which schema version, tool registration, policy revision, and verifier logic governed a call. Without that provenance, operators see symptoms downstream but cannot establish which boundary stopped agreeing.
I treat tolerant parsing cautiously here. Ignoring an unknown field can be compatible when that field is genuinely additive. Coercing an unfamiliar status into a familiar one is different: it hides a semantic disagreement. At control boundaries, an unrecognized value should usually fail closed or enter an explicit unknown state. Compatibility should be a deliberate property of the contract, not an accident of permissive code.
Contract Drift vs Verifier Drift
Contract drift concerns disagreement between connected components. Verifier drift concerns a check whose judgment has changed relative to the behavior it is supposed to assess. They can occur together, but they require different investigations.
| Question | Contract drift | Verifier drift |
|---|---|---|
| What changed? | An interface or shared behavioral expectation | The acceptance rule or its interpretation |
| Typical symptom | Producer and consumer assign different meaning to the same exchange | The same behavior receives a different pass or fail result |
| Primary control | Versioned contracts and boundary compatibility tests | Stable evaluation criteria and verifier regression tests |
| Main risk | Misrouted state, authority, or side effects | False acceptance or rejection |
A verifier can satisfy its current interface while applying the wrong standard. Conversely, two components can disagree about a result even when the verifier itself is stable. Separating these cases prevents teams from repairing a schema when the judgment moved, or retuning an evaluator when the integration contract is actually broken.
The Rifty take
We optimize for explicit incompatibility over plausible continuation. A broken boundary that stops the run is cheaper to recover from than a permissive boundary that silently changes authority, state, or the meaning of completion. We accept some versioning and test overhead to keep those disagreements visible.
Implementation checks
- Give schemas, tool registrations, and behavioral contracts stable identities or versions that appear in traces.
- Define the semantics of success, failure, retryability, partial completion, and unknown outcome alongside payload shapes.
- Test producer and consumer versions together, including mixed-version deployments and resumed runs created under an older contract.
- Reject unknown control values unless the contract explicitly defines forward-compatible handling.
- Verify side effects independently before converting an accepted request into a completed workflow step.
- Record the contract and policy versions used at each state transition.
- Include permission scope, credential expectations, and compensating actions in compatibility reviews.
- Exercise negative cases: missing fields, new enum values, reordered steps, stale registrations, duplicate delivery, and changed error classes.
- Ensure rollback logic is tested against every supported version of the state it may need to reverse.
- Remove obsolete contract versions deliberately; indefinite compatibility creates an expanding, poorly understood control surface.
Frequently asked questions
How is contract drift different from a breaking schema change?
A breaking schema change is one possible cause of contract drift, but drift is broader. Components may still exchange schema-valid data while disagreeing about status meanings, authority, retry rules, side effects, or completion evidence. Those semantic changes can be harder to detect because parsing continues successfully.
Can backward-compatible changes still cause contract drift?
Yes. A new optional field may be structurally backward compatible but operationally required for correct authorization or verification. Likewise, adding a status value can break consumers that map unknown values to a default. Compatibility must cover behavior and control decisions, not only whether old parsers accept the payload.
Where should a harness detect contract drift?
Detect it at component boundaries and during integration testing. Validate contract identities, supported versions, control values, permissions, and state-transition preconditions before work proceeds. End-to-end traces should also record which versions were active so an operator can distinguish drift from model error or provider failure.
Should an agent harness tolerate unknown contract values?
Only when the contract explicitly defines safe forward-compatible behavior. Unknown descriptive metadata may be ignored, but unknown statuses, permissions, action types, or completion states should normally fail closed or become an explicit unknown outcome. Guessing at control semantics turns an integration issue into an operational risk.
What is the most useful regression test for contract drift?
Use boundary tests that run real producer outputs through supported consumer versions and assert resulting state transitions, authority decisions, side-effect verification, and retry behavior. Shape-only fixtures are insufficient. The test should prove that both components assign the same operational meaning to the exchange.