How it works
Compaction is a control loop that runs alongside the agent loop. It watches token usage and intervenes before the context window overflows, so the session can keep going instead of erroring out or truncating from the front blindly.
A typical cycle looks like this:
- Measure. After each turn, compare current token usage against a budget threshold (often a fraction of the window, not the hard limit).
- Select a target. Choose a span to compress — usually the oldest turns, or large tool outputs above a size cap. Recent turns and the system prompt are normally kept verbatim.
- Summarize. Send that span to a model — sometimes a cheaper one — with instructions to preserve decisions, open tasks, and facts the agent will need later.
- Splice. Replace the raw span with the generated summary in place, reclaiming budget.
- Continue. Resume the loop with headroom restored.
Common variants include a rolling running-summary that grows as the session does, hierarchical summaries of summaries, and eviction of tool results that can be re-fetched on demand rather than summarized. The important property in all of them is that after compaction the agent no longer sees the original bytes. It sees a reconstruction it now treats as ground truth.
Why it matters in an agent harness
Compaction is where a harness quietly decides what the agent is allowed to forget. That makes it a correctness surface, not just a cost optimization.
The failure it introduces is not a crash. It is an agent that confidently proceeds on an incomplete picture — a file path that got summarized away, a constraint the user stated forty turns ago, an error it already hit and is about to hit again. The summary reads as authoritative, so the agent has no signal that anything was lost.
Three harness properties are directly affected:
- Observability. If you log only the compacted context, you lose the ability to reconstruct why the agent did something. Keep the raw transcript out of band; compaction is for the model's window, not for your audit trail.
- Reproducibility and evaluation. Compaction is model-generated and therefore non-deterministic. Two runs of the same task can diverge because they summarized different spans. If your evals don't account for this, you will chase phantom regressions.
- Failure containment. A bad summary can strip out exactly the guardrail context that was keeping the agent bounded — the note that said "don't touch production," now compressed into "working on deployment."
The practical stance: treat the compaction step as an untrusted transformation of state, and design so that the load-bearing facts don't live only in the part of the window that gets summarized.
Compaction vs checkpointing
These are often confused because both fight session length, but they solve different problems and one cannot substitute for the other.
| Context compaction | Checkpointing | |
|---|---|---|
| Goal | Fit within the context window | Recover or resume a session |
| Fidelity | Lossy summary | Lossless snapshot |
| Recoverable | No — original bytes are gone | Yes — restore exact state |
| Determinism | Model-generated, non-deterministic | Deterministic |
| Used for | Keeping the loop alive | Rollback, resume, audit |
The design decision this changes: never rely on compaction as your durability story. If a session dies mid-task and all you kept was the compacted window, you cannot faithfully resume. Pair compaction (to survive the window) with checkpointing (to survive the run).
The Rifty take
We optimize for legibility over runway. A summary the operator cannot audit is a silent state mutation, and we treat it as one. The tradeoff we accept is spending tokens and engineering effort to keep the raw transcript recoverable out of band, and to pin load-bearing facts — constraints, active goals, hard boundaries — somewhere compaction can't quietly rewrite them. Forgetting is a harness decision, so it should be an explicit one.
Common failure modes
- Silent detail loss. A path, ID, or constraint gets summarized away and the agent proceeds as if it never existed.
- Boundary erosion. Safety context ("read-only," "don't deploy") is compressed into something weaker, loosening the effective guardrail.
- Summary drift. Rolling summaries accumulate small distortions over many compactions until the working context is subtly wrong.
- Repeated work. The record of a failed attempt is dropped, so the agent retries the same dead end.
- Untraceable behavior. Only the compacted window is logged, so post-hoc debugging can't reconstruct the real cause.
- Eval instability. Non-deterministic compaction makes runs diverge, producing regressions that aren't real.
- Over-aggressive thresholds. Compacting too early throws away context that still fit, trading correctness for headroom you didn't need.
Frequently asked questions
When should an agent trigger context compaction?
Trigger it at a budget threshold below the hard window limit, not at overflow, so there's room for the summarization call itself. Compact the oldest turns and oversized tool outputs first, keep recent turns and the system prompt verbatim, and avoid compacting so early that you discard context that still fit.
How is context compaction different from checkpointing?
Compaction is a lossy summary that reclaims window space so the loop keeps running; checkpointing is a lossless snapshot that lets you restore exact state. Compaction cannot recover lost bytes, so it is not a durability mechanism. Use compaction to survive the window and checkpointing to survive the run.
How do I prevent compaction from dropping details the agent needs?
Keep load-bearing facts — active goals, hard constraints, safety boundaries — outside the span that gets summarized, for example pinned in the system prompt or a structured scratchpad. Retain the raw transcript out of band for audit, and let evictable tool results be re-fetched on demand rather than summarized into prose.
Does compaction hurt reproducibility and evals?
Yes. Compaction is model-generated and non-deterministic, so two runs of the same task can diverge because they summarized different spans. Account for this in evaluation by controlling or logging compaction, and don't attribute a run-to-run difference to a code change until you've ruled out the summarizer as the cause.
Is context compaction the same as context engineering?
No. Context engineering is the broader discipline of deciding what enters the window and how it's structured; compaction is one runtime tactic within it for shedding tokens under pressure. Good context engineering reduces how often compaction is needed and limits the damage when it runs.