How it works
Observability treats a running agent as a system that emits evidence. Each turn of the agent loop produces signals: the prompt sent, the model's reasoning, the tool it selected, the arguments, the result, and the state transition that followed. You capture those signals as structured traces and store them so a run can be reconstructed after the fact — not guessed at from a final answer.
A useful agent-observability stack collects three layers:
- Traces — the ordered sequence of loop iterations and tool calls, with inputs and outputs, tied to one run ID.
- State — the context window, memory, and external side effects (files written, rows changed) at each step.
- Signals — latency, token cost, error classes, and eval or guardrail verdicts attached to individual spans.
The key move is correlation. Every span carries the run ID and step index, so you can walk a failure backward from a bad output to the exact tool call that produced it. Because agents are stochastic, you record the trajectory that actually happened, not the one you intended. The instrumentation should be emitted by the harness around the loop, not bolted onto the model — that way it survives model swaps, prompt edits, and refactors.
Why it matters in an agent harness
An agent you cannot observe is an agent you cannot operate. Everything else the harness offers — control, reversibility, permissions — depends on being able to see what happened.
Observability pays off in four concrete places:
- Diagnosis. When a run goes wrong, a stored trajectory turns "the agent did something weird" into "step 7 called the wrong tool with a hallucinated argument." Without it, every failure is unreproducible by definition.
- Failure containment. Traces are where you notice a loop thrashing, a tool being retried forever, or a blast radius widening before the damage compounds. Alerts on span-level signals are how you cut a run early.
- Evaluation. Trajectory-level traces are the raw material for evals and for judging whether a change actually improved behavior. You cannot score a run you did not capture.
- Reversibility. Knowing which side effects a step produced is what lets you reason about a rollback. Observability tells you what to undo; the checkpoint lets you undo it.
The deeper point: model output is non-deterministic, so you cannot certify correctness by inspecting code paths the way you would with a normal service. The trace is your ground truth about what the agent chose, and it is the only durable record after the process exits.
Observability vs monitoring
The two are often conflated, but they answer different questions and that changes what you build.
| Monitoring | Observability | |
|---|---|---|
| Answers | "Is it up and within SLO?" | "Why did this run behave this way?" |
| Questions | Known in advance | Asked after the fact |
| Unit | Aggregate metrics | A single reconstructable run |
| Fit for agents | Catches outages | Catches wrong-but-successful behavior |
Monitoring tells you the service returned 200s. Observability tells you the agent returned 200s while quietly deleting the wrong records. For agents, most real failures are semantic, not availability failures — the run "succeeds" and still does the wrong thing. Dashboards alone will not surface that; per-run traces will.
The Rifty take
We instrument the harness, not the model. The model is a component we expect to swap, so the durable observability layer lives around the loop — spans on every tool call, state transition, and guardrail verdict, keyed to a run ID. We optimize for reconstructability over dashboard density: the test is whether an operator can open one failed run cold and explain it without rerunning it. We accept the storage and instrumentation cost as the price of being able to operate an autonomous system at all.
Common failure modes
- Logging outputs, not decisions. Capturing the final answer but not the tool calls and arguments leaves you unable to explain how the agent got there.
- No run correlation. Spans without a shared run ID and step index cannot be reassembled into a trajectory, so cross-step causation is lost.
- Instrumenting the model instead of the harness. Observability tied to a specific provider or prompt breaks the moment you route to a different model.
- Sampling away the interesting runs. Aggressive trace sampling tends to drop exactly the rare, long, expensive trajectories where failures hide.
- Recording intent, not reality. Storing the planned trajectory instead of the executed one hides the non-determinism you most need to see.
- Traces you never replay. Captured trajectories that are never fed into evals or postmortems are cost without payoff — observability only matters if someone reads it.
Frequently asked questions
How is agent observability different from application observability?
Agent observability adds the trajectory — the sequence of model decisions, tool calls, and state transitions — to the usual latency, error, and cost signals. Standard app observability assumes deterministic code paths; agents are stochastic, so you must record which choices actually happened, not just that a request completed.
What should I trace on every agent run?
Capture the ordered loop steps with a shared run ID: each prompt, the selected tool and its arguments, the tool result, the resulting state change, and any guardrail or eval verdict. Attach latency, token cost, and error class per span so a single failed run can be reconstructed cold, without rerunning it.
Why isn't monitoring enough for agents?
Monitoring answers questions you defined in advance — uptime, latency, error rates. Most agent failures are semantic: the run succeeds while doing the wrong thing. That behavior never trips an availability alert. You need per-run traces to ask new questions after the fact and to catch wrong-but-successful trajectories.
How does observability support reversibility?
Observability records which side effects each step produced — files written, rows changed, calls made. That record tells you precisely what to undo when a run goes bad. It pairs with checkpoints, which provide the mechanism to undo it; together they let an operator reverse an agent's actions with confidence rather than guesswork.
Should observability live in the model or the harness?
In the harness. The model is a swappable component, so instrumentation tied to a specific provider or prompt breaks on the next model swap. Emitting spans around the agent loop keeps your traces stable across model changes, refactors, and prompt edits, which is where durable operability comes from.