How it works
Diff review sits between an agent's proposed change and the state it wants to modify. The agent doesn't write directly to the source of truth; it writes to a staging layer, and the diff is the artifact you inspect.
A typical loop looks like this:
- The agent proposes changes against a known base — a git commit, a document version, a config snapshot.
- The harness computes a diff: the precise added, removed, and modified lines, scoped to the files actually touched.
- A reviewer — human or a second agent — reads the diff against intent, not just against "does it run."
- The change is accepted, amended, or rejected. Only accepted diffs advance the base.
The unit that matters is the hunk: a localized, readable change with enough surrounding context to judge it. A good diff review reads mechanism, not vibes — what state changed, whether the edit matches the stated goal, and what the change touched that it shouldn't have. The base version is the anchor. Without a stable, named base, a "diff" is just a snapshot, and you lose the ability to say precisely what the agent did versus what was already there.
Why it matters in an agent harness
Agents are stochastic and confident. They will rename a symbol you didn't ask about, delete a guard clause it decided was redundant, or rewrite a whole file when you wanted one line. Prose summaries hide this; a diff does not. Reading the exact changes is the cheapest way to catch drift between what the agent said and what it did.
Diff review is where reversibility becomes concrete. If every change flows through a reviewable diff against a base, you always have a clean rollback point and a clear record of what entered the system. It converts an opaque autonomous edit into an inspectable, contestable proposal — the same contract you'd hold a human contributor to.
It is also your last line before blast radius grows. A reviewed diff bounds the damage of a bad edit to the moment before merge. This is the difference between an agent that proposes and an agent that commits: the former can be wrong safely, the latter cannot.
Diff review vs approval gate
These are often conflated, but they answer different questions.
| Diff review | Approval gate | |
|---|---|---|
| Question | What exactly changed? | May this action proceed? |
| Object | The concrete change set | A pending operation or permission |
| Timing | After work, before merge | Before work, or before an effect |
| Failure it catches | Wrong or over-broad edits | Unauthorized or unsafe actions |
An approval gate can fire before the agent knows what it will produce. Diff review is inherently after production — you can't read a change that doesn't exist yet. The strongest harnesses use both: a gate to bound what an agent is allowed to attempt, and diff review to verify what it actually did. Approving a category of action is not the same as approving the specific bytes it wrote.
The Rifty take
We treat the diff, not the summary, as the reviewable object. An agent's explanation of its change is evidence, not truth; the change is truth. So we optimize for diffs that are small, scoped, and anchored to a named base — because a diff you can't read in one sitting is a diff you'll rubber-stamp. We accept the friction of a review step because the alternative is an agent whose edits you can only audit after they've already shaped the project. Legibility before merge is the boundary we hold.
Common failure modes
- Diffs too large to read. A 900-line change gets skimmed, not reviewed. Constrain agents to small, single-purpose changes so each diff is legible.
- Reviewing prose instead of the change. Trusting the agent's summary of what it did defeats the point. Read the hunks.
- No stable base. If the agent edits a moving target, the diff mixes its work with concurrent changes and you can't attribute intent. Pin the base.
- Approving structure, ignoring content. "Three files changed" tells you nothing about whether a guard clause vanished. Read what changed inside each file.
- Rubber-stamp fatigue. High volume plus low stakes trains reviewers to click accept. Route only meaningful diffs to a human; let deterministic checks clear the trivial ones.
- No amend path. If review is accept-or-reject only, reviewers accept flawed changes to avoid restarting. Allow targeted edits to the diff before merge.
- Reviewing without running. A diff can read clean and still break behavior. Pair review with an eval or test pass on the proposed state.
Frequently asked questions
How is diff review different from reading the agent's summary of its changes?
The summary is the agent's account of what it intended; the diff is what it actually wrote. Agents drift between the two — renaming symbols, deleting guards, rewriting whole files. Reading the diff catches that gap. Treat the summary as a hint and the change set as the reviewable truth.
Should a human or another agent perform diff review?
Both, layered by stakes. Route small, low-risk diffs through deterministic checks or a second agent to clear the trivial ones, and reserve human review for changes that touch high-blast-radius surfaces. Sending every diff to a human causes rubber-stamp fatigue; sending none removes your last legibility point before merge.
Why does diff review need a stable base version?
A diff is only meaningful against an anchor. Without a pinned base — a commit, document version, or config snapshot — you can't separate the agent's work from concurrent changes, and you lose clean attribution and rollback. The base is what makes a change set precise instead of just a snapshot.
How do I keep agent diffs small enough to actually review?
Constrain the agent to single-purpose changes and bound its scope per step. Large diffs get skimmed, not read, so a 900-line change is effectively unreviewed. Break work into small, anchored proposals, and require each to justify what it touched beyond the stated goal.
Is diff review enough to make agent changes safe?
No — it verifies what changed, but not that the new state behaves correctly. Pair it with an eval or test pass on the proposed state, and with an approval gate that bounds what the agent may attempt. Review catches wrong edits; gates and evals catch unsafe actions and broken behavior.