How it works
An execution journal turns workflow progress into durable state rather than leaving it inside a process, model context, or transient log stream. Before or around each consequential action, the harness writes a journal entry that identifies the workflow, step, attempt, intended operation, and current status. After the action returns, the harness records the result or failure and any external identifiers needed for later reconciliation.
A typical control sequence is:
- Create a stable key for the workflow step or logical operation.
- Record that the operation is planned or started before invoking the tool.
- Perform the action within its permission and execution limits.
- Record completion, failure, or an unknown outcome with the available evidence.
- On recovery, inspect the journal before choosing to skip, retry, reconcile, or compensate.
The ordering matters, but status transitions matter more. A process can fail after an external system accepts a request but before the harness records success. That creates an unknown outcome, not an ordinary failure. The journal must preserve that ambiguity. Recovery code can then query the external system using a request or resource identifier instead of repeating the action immediately.
The journal should represent authoritative workflow progress. Diagnostic logs and model transcripts may enrich it, but they should not be required to decide whether a step is safe to run again.
Why it matters in an agent harness
Agent runs cross boundaries that ordinary in-process control flow cannot protect. A model proposes work, the harness invokes tools, and those tools may change files, publish content, send messages, or update remote records. The model can be retried, the worker can restart, and a network response can disappear even when the effect occurred. Without a durable account of completed work, recovery becomes guesswork.
An execution journal gives the harness a control surface for that uncertainty. A resumed run can distinguish four materially different states: a step never started, a step currently leased to a worker, a confirmed failure, and an outcome that must be reconciled. Treating all four as “run it again” can duplicate irreversible effects.
The journal also makes retries conditional. A read-only lookup may be repeated freely. A write with a stable idempotency key may be retried under a defined policy. A non-idempotent action with an unknown outcome should stop at reconciliation or an approval boundary. The model should not improvise this distinction from conversation history.
This improves observability without making observability the source of truth. Operators can see which step owns the run, how many attempts occurred, why recovery chose a particular path, and whether a compensating action was required. Evaluators can inspect execution behavior rather than judging only the final answer.
The tradeoff is additional state machinery. Entries need stable identities, constrained transitions, durable storage, and concurrency control. That cost is justified when actions have side effects or workflows must survive interruption. For a disposable, read-only calculation, a journal may add ceremony without meaningful recovery value.
Execution journal vs audit trail
The two records overlap, but they answer different operational questions. Designing one vague event stream to serve both purposes usually weakens recovery semantics.
| Axis | Execution journal | Audit trail |
|---|---|---|
| Primary question | What should the workflow do next? | Who or what did what, and under which authority? |
| Main consumer | Recovery and orchestration logic | Operators, reviewers, and compliance processes |
| Required detail | Step identity, status, attempt, effect identifiers, recovery decision | Actor, action, target, time, permission context, and evidence |
| Mutation model | May advance through controlled status transitions | Usually append-oriented and resistant to alteration |
| Failure sensitivity | Must represent unknown outcomes explicitly | Must preserve an attributable record of observed events |
A single event can feed both records, and one storage system can support both projections. The design boundary is semantic: recovery must not depend on an audit record that lacks workflow state, while accountability must not depend on mutable status rows that erase earlier attempts.
The Rifty take
We treat the execution journal as part of the workflow's control contract, not as enhanced logging. We accept the cost of explicit state transitions because ambiguous retries are more expensive once an agent can cause external effects. If the harness cannot prove an action failed, it should record uncertainty and reconcile before repeating it.
Implementation checks
- Give every workflow, step, logical operation, and attempt a stable identifier.
- Write intent before invoking a consequential external action.
- Define allowed status transitions and reject impossible or backward transitions.
- Record external request and resource identifiers needed to reconcile unknown outcomes.
- Separate retryable failure from an action whose outcome is unknown.
- Make concurrent workers acquire ownership through a lease, lock, or equivalent control.
- Keep payloads minimal and protect secrets from journal storage.
- Test crashes before dispatch, during execution, and after the effect but before completion is recorded.
- Verify that recovery decisions are deterministic from journal state and current external evidence.
- Preserve enough history to explain skipped steps, retries, compensation, and manual intervention.
Frequently asked questions
Is an execution journal just a structured log?
No. A structured log describes events for diagnosis, while an execution journal holds the workflow state used to make recovery decisions. Logs may be incomplete, reordered, or unavailable to orchestration code. Journal entries require stable step identities, controlled status transitions, and enough effect information to determine whether another attempt is safe.
What should an execution journal record for each action?
Record the workflow and step identifiers, logical operation key, attempt number, status, timing, tool or action type, and recovery-relevant result identifiers. Include the reason for failure or uncertainty, but avoid storing credentials or unnecessary payloads. The test is whether recovery can choose its next action without reconstructing state from prose.
How does an execution journal make retries safer?
It makes retries conditional on recorded state and action semantics. The harness can skip confirmed work, retry a known failure under policy, or reconcile an unknown outcome using an external identifier. This prevents a restarted agent from treating missing context as permission to repeat a potentially irreversible operation.
How is an execution journal related to checkpointing?
Checkpointing preserves resumable state at selected boundaries; an execution journal records the action-level progression that explains how the workflow reached those boundaries. A checkpoint can restore context or computed data, while the journal determines which effects already occurred. Durable workflows often need both because restored context alone cannot prove side-effect completion.
When is an execution journal unnecessary?
It may be unnecessary for short, read-only, disposable operations whose complete computation can be safely repeated. It becomes valuable when a workflow spans process restarts, invokes costly tools, crosses external boundaries, or performs non-idempotent writes. The decision should follow recovery risk, not workflow length or the number of model turns.