How it works
A long agent task produces more tokens than any context window holds: tool outputs, intermediate reasoning, retrieved documents, prior turns. Context management is the control loop that decides what stays verbatim, what gets summarized, and what gets dropped — turn by turn, before the window overflows.
In practice a harness runs some version of this loop:
- Measure the current token budget against the model's window and a safety margin.
- Rank what is in context by relevance to the active goal, not by recency alone.
- Compact low-value spans — verbose tool logs, resolved sub-steps, superseded plans — into short summaries.
- Preserve load-bearing state verbatim: the task spec, open constraints, decisions already made, and identifiers the agent will need again.
- Re-inject compacted or retrieved state at the top of the next turn so the thread survives the cut.
The hard part is step 2 and step 4. Trim the wrong span and the agent forgets a decision it made an hour ago and re-litigates it. Keep everything and you exhaust the window mid-task. Good context management treats the window as a scarce, actively-managed resource rather than an append-only log.
Why it matters in an agent harness
Context is the agent's entire working memory. When you manage it badly, every downstream property of the harness degrades.
Reliability drops first. An agent that loses the original task spec starts optimizing a drifted version of the goal. You see it as tasks that end plausibly but off-target — the model finished a job, not the job.
Observability suffers. If the harness silently drops tool outputs, your trace no longer explains why the agent acted. You cannot debug a decision whose evidence was garbage-collected out of context.
Cost and latency compound. Uncompacted context means every turn re-processes the whole history. Token spend and time-to-response climb linearly with task length, which is exactly the regime — long tasks — where agents are supposed to earn their keep.
Failure containment weakens. Poisoned or injected content that enters context tends to persist through naive compaction. A summary can launder a prompt injection into a durable "fact" the agent now trusts. Context management is where you decide what untrusted evidence is allowed to survive.
The point is not to cram more in. It is to keep the right state legible to the model at every step so the rest of your controls — evals, approvals, rollback — are reasoning about an agent that still knows what it is doing.
Context management vs context engineering
These are often used interchangeably, but they operate at different times and change different decisions.
| Context management | Context engineering | |
|---|---|---|
| When | Runtime, per turn | Design time, before the run |
| Question | What do we cut or keep now? | What should the model ever see? |
| Scope | Compaction, trimming, re-injection | Prompt structure, retrieval, tool descriptions, formatting |
| Failure it prevents | Overflow, drift, lost thread | Irrelevant, malformed, or missing context |
Context engineering decides the shape of what enters the window; context management decides what survives as the task runs long. You need both — well-engineered context still overflows on a multi-hour task, and aggressive management cannot rescue context that was badly assembled to begin with.
The Rifty take
We treat the context window as state under the harness's control, not the model's. Compaction is a checkpointed operation: we would rather anchor the task spec, open constraints, and prior decisions as protected, non-summarizable state and aggressively trim the noisy middle than let a model "remember" freely and drift. We accept the tradeoff that some detail is lost at each compaction — but only detail the agent can re-derive or re-retrieve, never a decision it already made. When in doubt, isolate the work in a subagent with a fresh window instead of stuffing more into one thread.
Common failure modes
- Silent decision loss: a summary drops a choice the agent made, and it reopens a settled question. Protect decisions as verbatim state.
- Recency bias: trimming by age instead of relevance evicts the task spec while keeping chatty recent tool logs.
- Summary drift: each compaction summarizes the previous summary, and meaning erodes across generations. Re-anchor from source state, not from prior summaries.
- Injection persistence: untrusted content survives compaction as an asserted fact. Mark external evidence as untrusted and prefer to re-retrieve over carrying it.
- Unlogged cuts: the harness drops context without recording it, so your trace no longer explains the agent's behavior. Log what was compacted and when.
- Compacting past a checkpoint: trimming state the agent needs to resume after a rollback. Keep resume-critical state outside the compaction budget.
Frequently asked questions
When should a harness compact context?
Compact before the window fills, not after it overflows. Trigger on a token budget crossed against a safety margin, or at natural task boundaries — a completed sub-step or a checkpoint. Compacting reactively at overflow forces rushed, lossy cuts; compacting proactively lets you choose what to preserve deliberately.
Does context management make an agent forget its decisions?
It can, and that is the primary failure mode. Naive compaction trims by recency and evicts decisions the agent still needs, so it re-litigates settled questions. The fix is to treat the task spec, open constraints, and prior decisions as protected state that is never summarized away, and trim only re-derivable detail.
How is context management different from a bigger context window?
A bigger window delays the problem; it does not solve it. Long tasks still exceed any fixed budget, and stuffing a full window degrades reasoning and raises cost and latency every turn. Context management keeps the relevant state legible regardless of window size — it is a control loop, not a capacity you buy.
What state should never be summarized away?
Anything the agent cannot re-derive or re-retrieve: the original task spec, unresolved constraints, decisions already made, and identifiers needed to resume after a rollback. Everything else — verbose tool logs, resolved sub-steps, superseded plans — is a candidate for compaction. Protect the load-bearing state; trim the noisy middle.
Can compaction hide a prompt injection?
Yes. Summarizing untrusted content can launder an injected instruction into a durable 'fact' the agent then trusts, because the summary drops the signal that it was external evidence. Mark source excerpts as untrusted, prefer re-retrieval over carrying them, and never let a compaction step promote untrusted text to asserted state.