How it works
Agent tracing instruments the execution path of an agent rather than recording only its final response. The harness assigns an identity to the run, then emits linked events as work moves through the agent loop. Each event captures what occurred, when it occurred, which component initiated it, and how it relates to earlier events.
A useful trace usually follows this sequence:
- Record the run inputs, applicable policy, model selection, permissions, and execution limits.
- Capture each model call with its assembled context, response metadata, and place in the loop.
- Record proposed and completed tool invocations separately, including arguments, authorization decisions, results, errors, and retries.
- Link guardrail checks, approval gates, handoffs, checkpoints, and stop conditions to the event that triggered them.
- Close the run with its outcome, unresolved work, authoritative state, and termination reason.
The links matter as much as the events. A flat log can show that a tool failed and a retry followed. A trace should show that the failure caused the retry, which policy allowed it, and whether the second invocation reused or changed the arguments.
Tracing does not require storing every raw value forever. Sensitive context, credentials, and large tool responses can be redacted, hashed, summarized, or referenced through controlled storage. The requirement is that these transformations are visible. An omitted value must not look like a value that never existed.
Why it matters in an agent harness
An agent can produce the right result through the wrong process. It might exceed its permission scope, ignore an approval decision, retrieve unsupported evidence, or repeat a side effect before eventually returning a plausible answer. Output-only monitoring misses these failures because it evaluates the destination without inspecting the route.
A trace makes the harness legible at the level where control is actually exercised. Operators can determine whether a tool call was proposed or executed, whether credentials were scoped correctly, whether a guardrail ran before the action it governed, and whether a stop condition ended the loop. That sequence is critical during incident review. Knowing that a write occurred is weaker than knowing which context prompted it, which policy admitted it, and what happened immediately afterward.
Tracing also supports safer recovery. If the trace distinguishes durable state changes from speculative reasoning, a runner can locate the last confirmed checkpoint and decide whether to resume, retry, compensate, or stop. Without that distinction, an unknown outcome becomes dangerous. Repeating a read may be harmless. Repeating a payment, publication, message, or deletion may not be.
Evaluation becomes more diagnostic as well. A behavioral evaluation can test whether the final result is acceptable, while trajectory evaluation can identify where the run became inefficient or unsafe. Repeated retrieval, oscillation between tools, approval requests issued too late, and guardrails that never affect execution are trajectory problems. Traces expose them as patterns rather than isolated anecdotes.
There are costs. Detailed traces consume storage, increase privacy exposure, and can create a false sense of completeness. A trace only represents what the harness was instrumented to record. If permission resolution happens outside the traced boundary, the trace cannot prove that authorization was correct. Treat coverage as an explicit contract: define required event types, validate their presence, and surface instrumentation gaps as failures or warnings.
Trace access is itself a permission surface. Model inputs and tool results may contain private data, operational secrets, or adversarial content. Retention, redaction, and access controls should therefore be designed with the tracing schema, not added after traces become a convenient debugging archive.
Agent tracing vs audit trail
The distinction changes what you build. Agent tracing is optimized for reconstructing execution and diagnosing behavior. An audit trail is optimized for durable accountability around consequential events. One system may feed the other, but they should not be treated as interchangeable.
| Axis | Agent tracing | Audit trail |
|---|---|---|
| Primary question | How did this run unfold? | Who or what performed a consequential action? |
| Typical detail | Model calls, context assembly, tool attempts, retries, branches, and handoffs | Approved actions, state changes, identities, policy decisions, and timestamps |
| Retention | Often shorter and volume-sensitive | Usually governed by accountability and compliance needs |
| Mutability | May permit controlled enrichment or derived views | Should preserve durable, tamper-evident records |
| Main consumer | Engineers, evaluators, and operators | Operators, reviewers, and governance processes |
Do not copy an entire verbose trace into an audit store by default. Instead, promote the small set of consequential events that require durable accountability, while preserving identifiers that allow an authorized investigator to correlate them with the originating trace.
The Rifty take
We optimize tracing for reconstruction, not maximal collection. A useful trace makes control decisions and side effects unambiguous, even when some payloads must be redacted. We accept incomplete visibility into private reasoning, but not ambiguity about permissions, external actions, checkpoints, or termination.
Implementation checks
- Give every run, span, tool attempt, and handoff a stable identifier.
- Record proposed tool use separately from authorized and completed tool use.
- Preserve causal links across retries, branches, subagents, and asynchronous work.
- Capture the policy version, permission scope, execution limits, and stop condition applied to the run.
- Mark redacted, summarized, sampled, or externally stored fields explicitly.
- Distinguish model output from harness decisions and authoritative workflow state.
- Record side-effect outcomes precisely, including unknown outcomes that must not be retried blindly.
- Validate that required trace events exist before reporting the run as fully observable.
- Restrict trace access and retention according to the sensitivity of captured inputs and results.
- Test tracing during failures, because clean runs rarely exercise the events needed for recovery.
Frequently asked questions
What should an agent trace record?
An agent trace should record run inputs, context assembly, model calls, proposed and completed tool actions, permission decisions, guardrail results, retries, handoffs, checkpoints, state changes, and termination reasons. Each event needs stable identity and causal links so an operator can reconstruct execution rather than merely search disconnected logs.
Is agent tracing the same as logging?
No. Logging records events, while agent tracing connects events into the causal path of a run. Logs may reveal that an error and retry occurred. A trace should identify which action failed, why the retry was allowed, what changed between attempts, and how both attempts affected workflow state.
Should agent traces include prompts and model responses?
They should include enough information to explain model behavior, but raw prompts and responses are not always appropriate to retain. Sensitive values can be redacted, hashed, summarized, or stored behind tighter controls. The trace should explicitly mark that transformation so missing evidence cannot be mistaken for an absent event.
Can agent tracing make execution replay deterministic?
No. A trace can preserve inputs, decisions, tool results, and state transitions needed for investigation or controlled replay, but it cannot remove model non-determinism or changes in external systems. Replay must distinguish recorded results from newly executed actions and prevent consequential side effects from being repeated unintentionally.
How do you know whether tracing coverage is sufficient?
Define required event types from the harness control contract, then test whether normal, rejected, interrupted, retried, and partially completed runs emit them. Coverage is sufficient when operators can identify permissions, causal transitions, durable state, side effects, and termination reasons without inferring critical steps from a final response.