How it works
Determinism means a process is a pure function of its inputs: same inputs, same output, every time. It requires that you remove or fix every hidden input — the sources of variation that quietly change behavior between runs.
In practice, making a step deterministic is the work of eliminating those hidden inputs:
- Fix the seeds and clocks. Pin random seeds, freeze timestamps, and inject
now()rather than reading the wall clock inline. - Order the unordered. Sort map iterations, set stable tie-breaks, and pin concurrency so two runs interleave the same way.
- Pin the environment. Lock model IDs, tool versions, and dependency hashes so an upgrade doesn't silently alter results.
- Isolate external state. Reads from a mutable database or a live API are inputs too; snapshot or mock them when you need reproducibility.
The honest boundary in an agent harness is that the model itself is not deterministic. Even at temperature zero, sampling, batching, and hardware differences make token output variable. So determinism is not a property you extract from the LLM. It is a property you build into everything around it — the control flow, the tool layer, the verifiers, and the state transitions — while treating generation as the one stochastic component you have to contain.
Why it matters in an agent harness
Determinism is what lets you reason about an agent at all. If a run can't be reproduced, you can't debug it, can't test it, and can't trust that a fix actually fixed anything.
Concretely, it buys you four things. First, reproducible failures: a captured trajectory replays to the same state, so a bug is inspectable instead of anecdotal. Second, meaningful evals: a regression suite only means something if the harness scaffolding is stable, so a score change reflects the model or prompt and not incidental noise. Third, safe retries: when the deterministic parts of a step behave identically on replay, re-running after a crash converges instead of drifting into a new, divergent path. Fourth, auditability: a deterministic control path produces the same log for the same decision, which is what makes an approval or rollback trustworthy after the fact.
The design move that follows is to draw a hard line between the stochastic core and the deterministic shell. Let the model propose. Let deterministic code decide, validate, and commit. The more of the harness you can make replayable, the smaller the surface where surprises can originate.
Determinism vs idempotency
These get conflated, and the conflation produces broken retry logic. Determinism is about outputs matching across runs. Idempotency is about repeated application being safe. They are orthogonal, and you often want both.
| Property | Question it answers | Failure it prevents |
|---|---|---|
| Determinism | Do I get the same result every time? | Unreproducible bugs, noisy evals |
| Idempotency | Can I apply this twice without extra effect? | Double-charges, duplicate writes on retry |
A deterministic function can still be non-idempotent (it always returns the same value but each call appends a row). An idempotent operation can be non-deterministic (a create-or-update whose result depends on live state). A durable agent step usually needs a deterministic decision feeding an idempotent side effect.
The Rifty take
We treat the model as the only source of non-determinism in the system and push everything else toward being replayable. What we optimize for is a small, legible stochastic core wrapped in deterministic control flow, so that when something goes wrong the blast radius of "we can't explain this" is confined to the generation step. The tradeoff we accept is more scaffolding and less magic: fewer clever emergent behaviors, more boring, inspectable machinery. Reproducibility over cleverness is a boundary we enforce on purpose.
Implementation checks
- Replay a captured run and diff the trajectory; anything that differs is an unfixed hidden input.
- Grep for hidden inputs: direct
now(),random(),uuid(), and unordered iteration in control-flow code. - Pin versions explicitly — model IDs, tool and dependency hashes — so an upgrade is a deliberate change, not a silent drift.
- Separate decide from commit: keep the deterministic decision distinct from the idempotent side effect so retries are safe.
- Isolate the model boundary: mock generation in tests so you can assert on deterministic harness behavior without paying for stochastic output.
- Watch for false determinism: a step that passes locally but varies in production usually reads unpinned external state.
Frequently asked questions
Can an LLM agent ever be fully deterministic?
No. Even at temperature zero, sampling, batching, and hardware differences make token output variable across runs. Treat generation as the one stochastic component and build determinism into everything around it — control flow, tool calls, verifiers, and state transitions — so the unpredictable surface stays small and contained.
How is determinism different from idempotency?
Determinism means the same input yields the same output every run. Idempotency means applying an operation twice is safe. They are orthogonal: a deterministic step can still double-write, and an idempotent one can depend on live state. Durable agent steps usually pair a deterministic decision with an idempotent side effect.
Why does determinism matter for evaluating agents?
Evals only mean something if the harness scaffolding is stable. If control flow, tool versions, or environment vary between runs, a score change could reflect noise rather than the model or prompt. Pinning those hidden inputs makes a regression suite measure the change you care about instead of incidental variation.
What are the common hidden inputs that break determinism?
Wall clocks, random seeds, UUID generation, unordered map iteration, uncontrolled concurrency, unpinned model or dependency versions, and reads from mutable external state. Each is an input the process silently depends on. Injecting or pinning them turns a step into a pure function of its declared inputs, which is what replay requires.
How do you make retries safe when the model is non-deterministic?
Separate the deterministic decision from the idempotent side effect. Let the model propose, capture its output, then have deterministic code validate and commit through an idempotent operation. On retry the committed effect doesn't duplicate, and checkpointed state lets the run resume from its last stable point rather than regenerating a divergent path.