Compaction

How it works

An agent session grows monotonically. Every model turn, tool call, and tool result is appended to the message history, and that history is finite in a way the task is not. Compaction is the loop step that keeps the two reconciled.

The control loop is straightforward:

1. Measure. Track the token size of the working context after each turn.
2. Trigger. When usage crosses a threshold — a fraction of the window, or a hard turn count — fire compaction before the next model call.
3. Select. Split the history into a tail you keep verbatim (recent turns, the active task, the last few tool results) and a head you are willing to compress.
4. Summarize. Replace the head with a shorter representation: a running summary, extracted decisions, open questions, and any durable state — file paths, IDs, chosen approach.
5. Splice. Rebuild the context as system + summary + verbatim tail and continue.

The summary is itself usually produced by a model call, which means compaction is a lossy transform you are choosing to trust. What survives is a policy decision, not an accident: you decide what must be carried forward and what can be dropped. Done well, the agent notices nothing except a shorter prompt. Done badly, it silently forgets the constraint it agreed to three hundred turns ago.

Why it matters in an agent harness

Compaction is where a long-running agent either stays legible or quietly drifts. Without it, sessions hit a hard ceiling and either error out or start truncating from the front, dropping the original instructions first — the worst possible thing to lose. With naive compaction, the agent keeps running but loses the thread of what it was told, and you get confident action against a corrupted memory of the goal.

For the harness, three properties are at stake. Observability: a compaction event is a state transition, and if it isn't logged with what was dropped, you lose the ability to explain why the agent changed behavior mid-run. Reversibility: compaction is destructive to context, so the pre-compaction history should be checkpointed to durable storage, not thrown away — you want to reconstruct the full trajectory even though the model no longer sees it. Reliability: the summarizer can hallucinate, so it is another place a false fact can enter the loop and then get treated as ground truth on every subsequent turn.

The practical upshot: treat compaction as a first-class, instrumented step in the agent loop, not as a library default you never look at.

Compaction vs. truncation

Both keep the context under the window. They fail in opposite directions, and the choice changes what you can promise about correctness.

Truncation is cheap and predictable but forgets the beginning, which in an agent run is usually the instructions and constraints. Compaction preserves intent at the cost of a lossy, model-generated rewrite. Most durable harnesses pin the system prompt and task spec outside the compactable region, then compact the middle — you get the reliability of retention without paying to summarize the parts you can afford to keep verbatim.

The Rifty take

We treat compaction as a controlled edit to the agent's memory, not a housekeeping detail. Anything load-bearing — the task spec, the constraints, the approved plan — lives in a pinned region that compaction is not allowed to touch; only the disposable middle gets summarized. We accept the latency and token cost of an explicit summarizer over the silent forgetting of truncation, and we checkpoint the full pre-compaction history so a run stays reconstructable even after the model's view of it is gone.

Implementation checks

• Pin the system prompt, task spec, and hard constraints outside the compactable region.
• Log every compaction event: trigger reason, token counts before/after, and what was summarized versus kept.
• Checkpoint the full history before compacting so the trajectory stays reversible and auditable.
• Keep a verbatim tail of recent turns; don't summarize the turn the agent is mid-action on.
• Validate that durable state — IDs, paths, decisions — survives the summary; test with a fact planted early and checked late.
• Treat the summarizer's output as untrusted: it can hallucinate, so don't let it silently rewrite constraints.
• Budget the summarizer call itself; a compaction pass that keeps re-triggering signals a threshold set too tight.

Frequently asked questions

When should compaction trigger?

Trigger compaction before the next model call once context crosses a set fraction of the window — often a threshold well under the limit — or a hard turn count. Firing early leaves headroom for the summarizer's own output and the next tool result, avoiding a mid-turn overflow that forces abrupt truncation.

What is the difference between compaction and truncation?

Compaction summarizes older turns into a shorter form; truncation drops them outright. Compaction costs an extra model call and risks a lossy summary but preserves intent. Truncation is cheap and predictable but forgets the oldest content — usually the original instructions, which is the worst thing to lose in an agent run.

Can compaction cause an agent to forget its instructions?

Yes, if the instructions sit inside the compactable region. The fix is to pin the system prompt, task spec, and hard constraints outside compaction so only the disposable middle gets summarized. A lossy summarizer should never be trusted to faithfully re-encode the constraints the agent must still obey.

Is compaction reversible?

The context edit is destructive, but the session need not be. Checkpoint the full pre-compaction history to durable storage before you compress, so the complete trajectory stays reconstructable for auditing and debugging even though the model no longer sees it. Reversibility lives in the harness, not in the model's working context.

Does compaction risk introducing hallucinated facts?

It can. The summary is usually produced by a model call, so it is another place a false or distorted fact can enter the loop and then be treated as ground truth on every later turn. Treat summarizer output as untrusted, and test that planted early facts survive compaction intact.