How it works
Authoritative workflow state is the harness’s source of truth for execution decisions. It records durable facts such as the current workflow phase, completed steps, committed side effects, outstanding approvals, retry counts, leases, and allowed next transitions. A model may propose an action, and a worker may report an outcome, but neither report becomes authoritative merely because it exists.
A typical transition follows a guarded sequence:
- Read the current state and its version.
- Check that the requested transition is legal from that state.
- Reserve or identify the intended side effect with an idempotency key.
- Execute the work within the granted permission scope.
- Commit the outcome using a conditional write.
- Schedule the next legal transition or enter a defined recovery state.
The conditional write matters. If another worker changed the record after step one, the stale transition must fail instead of overwriting newer facts. For external side effects, the state should distinguish planned, attempted, confirmed, and unknown outcomes. Collapsing those states into a single completed flag makes recovery unsafe.
The authoritative record can be materialized state, an execution journal folded into state, or both. The storage design is secondary. What matters is that every controller agrees which record governs progress and that updates obey an explicit transition contract.
Why it matters in an agent harness
Agents generate plausible continuations. Workflow controllers need defensible facts. Without authoritative state, a harness can mistake a confident model message, a worker timeout, or a partial trace for evidence that work either succeeded or failed. The resulting retries may duplicate payments, messages, deployments, or file mutations.
A durable source of truth gives the harness control over non-deterministic execution. The model can choose among permitted actions, but it cannot redefine which actions already happened or which recovery path is legal. That boundary prevents conversation history from becoming an accidental transaction log.
Authoritative state also makes interruption and resumption tractable. A restarted worker does not need the original process memory. It reads the committed state, reconstructs the remaining work, and continues from a legal checkpoint. If an external call timed out after submission, the state can preserve an unknown outcome and require reconciliation rather than blindly retrying.
Observability improves because operators can distinguish workflow facts from diagnostic evidence. A trace may show that a tool call began. The authoritative record says whether its outcome was confirmed and incorporated into the workflow. That separation supports clear incident questions: What was committed? What remains uncertain? Which transition is blocked? What action can safely be replayed or compensated?
Permissions can also depend on state. An approval may authorize one specific transition from one version of a workflow, not grant a standing right to act. Once the state changes, stale approval tokens should no longer unlock the operation. This constrains delegated authority and reduces the blast radius of confused or duplicated workers.
The tradeoff is coordination cost. Durable writes, version checks, reconciliation, and explicit transition rules add latency and implementation work. I consider that cost justified wherever an action is expensive, externally visible, difficult to reverse, or capable of changing future permissions.
Authoritative Workflow State vs execution trace
These records often contain overlapping events, but they answer different questions. Treating a trace as state is a design error when diagnostic completeness is weaker than transactional correctness.
| Axis | Authoritative workflow state | Execution trace |
|---|---|---|
| Primary question | What is committed, current, and legally next? | What happened during execution? |
| Write semantics | Guarded by transition and concurrency rules | Commonly append-oriented diagnostic recording |
| Recovery role | Directly controls retry, resume, and compensation | Supplies evidence for investigation or reconstruction |
| Completeness requirement | Must cover every decision-bearing state | May be sampled, delayed, redacted, or partially missing |
| Safe controller input | Yes, by contract | Only if explicitly promoted through validation |
A trace event can support a state transition, but it should not silently cause one. The harness must validate the event, associate it with the correct workflow version and attempt, then commit the resulting state change. Conversely, authoritative state need not preserve every prompt fragment, token, or timing detail required for debugging. Keeping the roles separate lets each record have an honest reliability contract.
The Rifty take
We optimize for one explicit authority over progress, side effects, and recovery. We accept extra state-machine and persistence work to avoid letting transcripts, worker memory, or model confidence decide what is true. When an outcome cannot be confirmed, the honest state is unknown, followed by reconciliation or a bounded operator decision.
Implementation checks
- Name the single record or deterministic fold that controllers treat as authoritative.
- Define legal transitions and reject transitions from stale state versions.
- Separate intended, attempted, confirmed, failed, and unknown side-effect outcomes.
- Give externally visible actions stable idempotency keys tied to workflow identity.
- Bind approvals and permissions to a specific action, state, and version.
- Make retries consult committed state before repeating work.
- Define reconciliation paths for timeouts and ambiguous provider responses.
- Keep traces diagnostic unless validation explicitly promotes evidence into state.
- Test worker crashes before, during, and after the authoritative commit.
- Verify that two workers cannot both commit mutually exclusive transitions.
Frequently asked questions
Does authoritative workflow state require a relational database?
No. It requires a clearly designated source of truth with durable writes, concurrency control, and enforceable transition rules. A relational database is one option, but an append-only journal, durable workflow engine, or transactional key-value store can work if controllers derive the same state deterministically.
Can an execution log serve as authoritative workflow state?
Yes, but only when the log has a stronger contract than an ordinary diagnostic trace. Events must be durable, ordered or conflict-resolved, uniquely associated with attempts, and folded through validated transition rules. Sampled, delayed, or best-effort logs should never govern retries or committed side effects.
How should the harness record a timed-out external action?
Record the outcome as unknown when the request may have reached the external system but no reliable result returned. Do not translate uncertainty into failure. Reconcile using an operation identifier, provider lookup, or bounded operator check before retrying, compensating, or declaring the workflow complete.
What belongs in authoritative state instead of agent memory?
Store any fact needed to control progress, permissions, recovery, or side effects in authoritative state. This includes committed steps, attempt identities, approval status, retry limits, external operation references, and legal next transitions. Agent memory can retain useful context, but it should not decide transactional truth.
How do checkpoints relate to authoritative workflow state?
A checkpoint is a durable recovery point, while authoritative workflow state is the broader record that determines whether that checkpoint is valid and what may happen next. The state may reference checkpoint data, but restoring it must not erase later commits or revive permissions that have expired.