How it works
An audit trail turns an agent run into a sequence of attributable decisions. The harness records events at control boundaries, not only whatever the model happens to say. Useful events include tool requests, permission checks, approvals, denials, policy results, state transitions, retries, interrupts, and operator overrides.
Each event should answer a compact set of questions:
- What action or decision occurred?
- Which agent, operator, policy, or service initiated it?
- What authority and permission scope applied?
- Which run, task, and prior event did it belong to?
- What outcome was observed?
- Was any state or external system changed?
The record needs stable identifiers and ordering. Timestamps help, but timestamps alone are insufficient when work runs concurrently. Parent event IDs, sequence numbers, and run IDs preserve causality across workers and retries.
Sensitive inputs require deliberate handling. The trail can store hashes, references, classifications, or redacted summaries instead of raw credentials and private context. Redaction itself should be visible as metadata so an operator knows the record is incomplete by design.
For consequential actions, I prefer append-oriented events over mutable status rows. A current status is useful for scheduling. An event history explains how the system reached that status and whether the transition was authorized.
Why it matters in an agent harness
An agent can produce the right final state through the wrong process. It may exceed its delegated authority, call an overly broad tool, ignore a denial, or repeat a side effect after losing confirmation. Inspecting only the output hides those failures. An audit trail makes the harness's control decisions inspectable alongside the result.
That changes incident response. When an external write looks wrong, the operator can identify the initiating run, the permission decision, the approved arguments, the returned outcome, and any later retry. Without that chain, recovery becomes guesswork. The operator may not know whether to retry, compensate, revoke credentials, or stop related runs.
Auditability also strengthens approval systems. An approval should bind to a specific proposed action, argument set, scope, and validity window. The trail should show whether execution matched what was approved. Recording only “approved” leaves room for the agent to change the target or parameters after the human decision.
The same record supports evaluation, but auditability and evaluation are not identical. Evaluators can use trail events to detect policy bypasses, excess retries, unnecessary escalation, or tools invoked outside the expected path. The trail supplies evidence. The evaluation supplies judgment.
There is a real cost. Detailed records consume storage, complicate privacy controls, and can expose sensitive operational context. The answer is not indiscriminate logging. It is a defined event contract that captures authority, causality, and effects while minimizing raw secrets and unrelated content.
Audit trail vs execution trace
The distinction affects retention, schema design, and who must be able to rely on the record.
| Axis | Audit trail | Execution trace |
|---|---|---|
| Primary question | Was the action authorized and accountable? | How did execution behave internally? |
| Typical events | Approvals, denials, overrides, permission decisions, external effects | Model calls, spans, timing, intermediate steps, tool latency |
| Reliability expectation | Durable enough for operational review | Often optimized for debugging and performance analysis |
| Data posture | Selective, attributable, and retention-aware | Potentially high-volume and deeply diagnostic |
| Main consumer | Operators, reviewers, recovery logic | Developers, observability systems, evaluators |
One event can serve both purposes, but the systems should not be treated as interchangeable. A sampled trace may be acceptable for latency analysis and unacceptable as the only record of a destructive action. Conversely, an audit event can prove that a write was approved without containing enough internal detail to diagnose why the agent proposed it.
The Rifty take
We optimize for reconstructing consequential decisions, not recording every token. The boundary is simple: if an event changes authority, external state, recovery options, or operator control, the harness should preserve an attributable record. We accept some storage and schema discipline because silent ambiguity after a side effect is more expensive.
Implementation checks
- Assign stable IDs to runs, events, actors, approval requests, and tool calls.
- Record both requested actions and observed outcomes; do not infer one from the other.
- Bind approvals to exact arguments, targets, scopes, and expiry conditions.
- Preserve causality across retries, workers, subagents, and resumed runs.
- Distinguish denied, failed, timed-out, cancelled, and unknown outcomes.
- Make redaction and omitted fields explicit in event metadata.
- Restrict access to the trail and avoid storing credentials or unnecessary private context.
- Test whether an operator can reconstruct a failed external write without consulting ephemeral logs.
- Detect gaps in the event sequence instead of presenting a partial trail as complete.
Frequently asked questions
What should an agent audit trail record?
An agent audit trail should record consequential requests, policy decisions, approvals, denials, tool calls, observed outcomes, state transitions, retries, interrupts, and overrides. Each event should identify its actor, run, authority, permission scope, target, and causal parent while excluding credentials and unrelated private context.
Is an audit trail the same as agent observability?
No. An audit trail preserves attributable evidence about authority, decisions, and effects, while agent observability covers broader operational behavior such as latency, errors, resource use, and internal execution. The two can share events, but observability data may be sampled or transient in ways an operational audit record should not be.
Should an audit trail include model prompts and responses?
Only when they are needed to reconstruct a consequential decision and can be retained safely. Full prompts may contain secrets, personal data, or irrelevant context. Prefer structured decision fields, content references, hashes, and explicit redaction metadata unless exact text is required for a defined review or recovery purpose.
How does an audit trail help with retries and unknown outcomes?
It shows whether an action was requested, dispatched, acknowledged, failed, or left without a confirmed result. That distinction prevents a harness from treating missing confirmation as failure and blindly repeating a side effect. Combined with idempotency or reconciliation, the trail gives recovery logic evidence for its next decision.
Can audit trail events be edited or deleted?
Corrections should usually be represented as new attributable events rather than silent edits to prior history. Retention and privacy requirements may still require deletion or redaction. When that happens, the system should preserve visible metadata indicating that the record changed, why it changed, and under which authority.