Context Engineering

How it works

Context engineering treats the context window as a scarce, actively-managed resource rather than a bucket you keep pouring into. On every turn, something decides what the model sees: the system prompt, the available tool definitions, retrieved documents, prior messages, tool outputs, and scratch state. Context engineering is the discipline of making that assembly deliberate.

In practice it is a loop that runs before each model call:

1. Select. Choose which instructions, memories, and retrieved chunks are relevant to the current step — and exclude the rest.
2. Compress. Summarize or truncate history and large tool outputs so signal survives and volume does not.
3. Order. Place high-priority material where the model attends to it; keep the task and constraints legible near the edges of the window.
4. Isolate. Fence untrusted external text so it reads as data, not as instructions.
5. Budget. Cap tokens per source so no single tool result crowds out the task.

The constraint is visibility: the model can only act on what you let into the window, and everything you let in competes for attention and dilutes the rest. Good context engineering is as much about what you withhold as what you include.

Why it matters in an agent harness

An agent loop accumulates context by default — every tool call appends output, every step adds messages. Left alone, the window fills with stale results, redundant retrievals, and low-signal chatter until the model loses the thread, contradicts earlier steps, or blows the token limit mid-task. Context engineering is how the harness keeps the loop legible and controllable over long runs.

Concretely, it buys you:

• Reliability over length. Compressing and pruning history keeps a long-running agent coherent instead of degrading as the transcript grows.
• Failure containment. Isolating external text at the context boundary is a primary defense against prompt injection — untrusted content that never enters as an instruction cannot hijack the loop.
• Observability. When you own the assembly step, you can log exactly what the model saw on the turn that went wrong, which turns a mysterious output into a reproducible one.
• Cost and latency control. Token budgets per source keep spend and response time bounded and predictable.

The window is the surface the model reasons over. Engineering it is engineering the agent's behavior at its root.

Context engineering vs prompt engineering

The two are often conflated, but they operate at different layers and the distinction changes how you build.

Prompt engineering asks how do I phrase the ask? Context engineering asks what should be in the room when the model answers, and what must stay out? In an agent harness the second question dominates, because the prompt is static while the context is assembled fresh on every iteration.

The Rifty take

We optimize for legibility over recall. A window packed with everything that might be relevant is worse than a lean one holding only what this step needs — attention is finite and irrelevant context is a tax, not a hedge. We accept the tradeoff that aggressive pruning occasionally drops something useful, because a predictable, inspectable context is worth more than a comprehensive one you cannot reason about. The boundary we enforce hardest: untrusted content enters as data, never as instruction.

Common failure modes

• Context accumulation. History and tool outputs append without bound until the model degrades or the window overflows. Fix with compaction and per-source token budgets.
• Retrieval flooding. Dumping the top-k chunks in raw when two would do — dilutes attention and buries the answer.
• Instruction leakage. External text pasted inline gets read as a command; the injection defense is isolation at the boundary, not a hopeful disclaimer.
• Stale state. Old plans, resolved errors, or superseded results linger and contradict the current step.
• Invisible assembly. No log of what actually entered the window, so a bad turn cannot be reproduced or explained.
• Lost-in-the-middle ordering. Critical constraints placed where the model attends least. Order for attention, not for chronology.

Frequently asked questions

How is context engineering different from prompt engineering?

Prompt engineering shapes the wording of a single, mostly-static prompt. Context engineering governs the full set of tokens present on every turn — instructions, tools, retrieved data, and history — and recomputes that assembly each loop step. In an agent harness the second layer dominates behavior.

Why not just put everything relevant in the context window?

Attention is finite and every added token competes with the rest, so a bloated window buries the signal, invites contradiction, and drives cost and latency up. A lean, deliberately-selected context usually outperforms a comprehensive one, and it stays inspectable when a turn goes wrong.

How does context engineering defend against prompt injection?

By isolating untrusted external text at the context boundary so it enters the window as data, not as instruction. Content the model never reads as a command cannot hijack the loop. Visibility discipline — controlling exactly what is present and how it is framed — is the primary structural defense.

What keeps a long-running agent's context from overflowing?

Compaction and budgeting. Summarize or truncate history and large tool outputs so signal survives while volume does not, and cap tokens per source so no single result crowds out the task. This keeps a long loop coherent instead of degrading as the transcript grows.

Where should the most important instructions go in the window?

Order material for attention rather than chronology. Models attend less to content buried in the middle of a long window, so place the task, constraints, and high-priority context near the edges. Chronological append order is convenient but routinely puts critical instructions where the model reads them weakest.