How it works
An agent trace captures a run as a sequence of causally linked events. Each event should identify what the harness supplied, what the agent decided, which action it requested, what the environment returned, and how that result affected the next step. The trace is useful because it preserves the path, not merely the final output.
A practical trace usually records:
- The assembled context, active instructions, model, and relevant run configuration.
- Each model response, including proposed actions and stop decisions.
- Tool-call arguments, permission decisions, execution status, and attributed results.
- State transitions such as checkpoints, retries, approvals, interruptions, and resumptions.
- Stable identifiers that connect events across workers, tools, and parent or child runs.
The harness should emit these events at the boundary where actions are accepted or rejected. Logging only inside the model adapter misses permission checks and external effects. Logging only inside tools misses the decision that selected them.
A trace does not make a stochastic run deterministic. It makes the observed run inspectable. Exact replay may still fail when models, external data, clocks, or mutable systems have changed. For that reason, a strong trace separates recorded facts from reconstructed explanations and preserves unknown outcomes rather than guessing whether an interrupted action completed.
Why it matters in an agent harness
A final answer tells you what the agent produced. A trace tells you how the harness allowed it to get there. That difference matters whenever an agent can read private data, mutate state, spend a budget, invoke code, or delegate work.
For control, traces expose the actual permission path. An operator can see whether a sensitive tool call passed through the expected policy, whether credentials were scoped correctly, and whether an approval governed the exact arguments later executed. This turns a permission model from a design claim into something that can be inspected run by run.
For failure containment, a trace establishes the last known state before an error. If a process stops after requesting an external mutation but before receiving confirmation, the trace should mark an unknown outcome. The recovery path can then reconcile the external system before retrying. Without that evidence, a retry may duplicate work, while an assumed success may silently lose it.
For evaluation, traces reveal behavioral defects hidden by acceptable outputs. An agent may reach the right answer after ignoring a required source, making unnecessary tool calls, exceeding an iteration budget, or recovering accidentally from a malformed result. Outcome-only evaluation misses those weaknesses. Trajectory evaluation can test whether the agent used approved tools, respected boundaries, and followed a defensible path.
For observability, trace events let engineers compare failure patterns across runs. The useful unit is not an unstructured transcript dump. It is a typed event with timestamps, correlation identifiers, inputs, results, and state transitions. Sensitive values may need redaction or controlled access, but redaction should preserve enough structure to explain the decision and attribute the effect.
Trace completeness also has a cost. Capturing every token, payload, and intermediate artifact increases storage, privacy exposure, and review burden. The design decision is therefore not whether to record everything. It is which events are necessary to reconstruct authority, causality, state, and recovery.
Agent trace vs audit trail
An agent trace and an audit trail may share events, but they answer different operational questions. Treating them as interchangeable usually produces either an unreadable debugging stream or an audit record too weak to establish accountability.
| Axis | Agent trace | Audit trail |
|---|---|---|
| Primary question | How did this run progress? | Who or what changed an important resource? |
| Typical scope | Model steps, context, tool calls, results, and control decisions | Security- or business-relevant actions and their authorization |
| Detail level | High enough to diagnose behavior and evaluate a trajectory | Minimal, durable evidence needed for accountability |
| Retention and access | Often tuned for debugging and evaluation | Usually governed more strictly because it records consequential actions |
| Failure handling | Preserves retries, malformed results, and unknown outcomes | Preserves attributable attempts, approvals, denials, and effects |
A harness can derive selected audit events from its trace, but only if those trace events are stable, tamper-resistant enough for the intended use, and explicit about identity and authorization. A raw reasoning transcript is not automatically an audit trail. Conversely, an audit record that says a tool ran may be insufficient to diagnose why the agent selected it.
The Rifty take
We optimize traces for reconstructing control decisions and external effects, not for collecting the largest possible transcript. We accept selective omission of low-value internal detail, but not gaps around permissions, tool arguments, state transitions, or unknown outcomes. If a run cannot explain what authority produced an effect, its trace is incomplete.
Implementation checks
- Give every run, step, tool call, and delegated child run a stable identifier.
- Record the effective policy and permission decision at the point of action.
- Preserve tool arguments and results with explicit, visible redaction markers.
- Distinguish requested, accepted, started, succeeded, failed, denied, and outcome-unknown states.
- Link retries to the original attempt instead of presenting them as unrelated calls.
- Record checkpoints, resumptions, interrupts, stop conditions, and budget exhaustion.
- Keep timestamps and event ordering meaningful across workers.
- Test whether an operator can reconstruct a failed run without consulting transient process logs.
- Restrict trace access and retention according to the sensitive data it may contain.
- Verify that logging failures surface visibly rather than allowing an apparently untraced success.
Frequently asked questions
What should an agent trace record?
An agent trace should record the effective context, model decisions, tool requests, permission outcomes, tool results, state transitions, retries, delegation, and termination reason. Stable identifiers should connect these events. Sensitive payloads can be redacted, but the trace must retain enough structure to reconstruct causality and authority.
Is an agent trace the same as a model transcript?
No. A model transcript covers messages exchanged with the model, while an agent trace covers the wider run. It includes harness policies, tool execution, approvals, state changes, retries, and external effects. The transcript may be one trace component, but it cannot explain controls that operate outside the model conversation.
Can an agent trace support exact replay?
Not by itself. A trace can preserve inputs, actions, results, and state closely enough for diagnosis or controlled simulation, but exact replay depends on frozen models, deterministic tools, immutable data, and captured environment state. When those conditions are absent, replay should report divergence instead of implying equivalence.
How should traces handle sensitive information?
Apply access controls, retention limits, and explicit redaction at capture or storage boundaries. Preserve event type, attribution, policy decision, and payload shape where possible. Redaction must remain visible; silently replacing sensitive values with plausible defaults can make the trace misleading and undermine later diagnosis or evaluation.
How do you know whether an agent trace is complete enough?
Test whether an operator can identify the active instructions, authority used, actions attempted, effects confirmed, state reached, and safe recovery path. If the operator must infer a permission decision or guess whether an external mutation completed, the trace is not complete enough for reliable operation.