How it works
Session affinity binds a sequence of related requests to one server instance. A router identifies the session, remembers which instance owns it, and sends later requests there while the binding remains valid. The affinity key might come from a session identifier, a routing token, or another stable request attribute. That key should locate state, not grant authority.
A typical control loop is:
- The first request reaches an eligible instance.
- That instance creates or loads the session state.
- The routing layer records the session-to-instance mapping.
- Later requests carrying the same affinity key return to that instance.
- The binding expires, migrates, or is rebuilt when the session ends or the instance becomes unavailable.
The mechanism is simple until ownership changes. An instance can restart, become unhealthy, or disappear during a long agent run. The harness then needs a declared response: restore the session on another instance from durable state, restart from a checkpoint, reject the request, or report an unknown outcome. Quietly routing to a fresh instance with empty state is unsafe because the request may look successful while losing prior tool results, approvals, limits, or pending actions.
Affinity therefore solves request placement. It does not, by itself, provide persistence, replay, failover, or correctness.
Why it matters in an agent harness
Agent sessions accumulate more consequential state than a conventional conversational exchange. The active instance may hold assembled context, loop position, tool observations, approval status, retry counters, temporary credentials, and records of side effects. Sending the next turn elsewhere without transferring that state can change what the agent believes has happened.
Correct affinity preserves continuity. A tool result remains connected to the call that produced it. An interrupt reaches the worker that owns the active loop. A human approval can resume the operation it actually authorized. Iteration and tool budgets continue from their current values instead of resetting on another instance.
The operational risk is hidden coupling. If important state exists only in process memory, the session becomes as fragile as the owning instance. A failure can leave the harness unable to distinguish among three materially different outcomes: an action never started, an action completed but its response was lost, or an action remains in progress. Retrying all three cases in the same way can duplicate external effects.
Session affinity should therefore sit inside a broader recovery contract. Durable checkpoints or an execution journal should preserve the minimum state needed to determine what happened and continue safely. Idempotency keys or explicit reconciliation should protect side-effecting calls. The router should expose binding creation, rebinding, expiry, and failed delivery as observable events.
Affinity also must remain separate from authorization. Possession of a valid routing token may identify the instance holding a session, but it must not expand the caller’s permission scope. The destination still needs to verify identity, delegated authority, tool permissions, and approval conditions. Otherwise, the routing mechanism becomes a confused deputy: it faithfully delivers a request to privileged state without proving that the requester may use it.
Finally, affinity affects failure containment. A bad instance can corrupt every session pinned to it, while overloaded sessions can make that instance a hotspot. Health checks and rebalancing policies need to account for state ownership rather than treating workers as interchangeable.
Session affinity vs stateless servers
These are design choices about where continuity lives, not direct opposites. A stateless server reconstructs each request from shared or supplied state. Session affinity permits continuity to depend on a particular instance, at least between persistence points.
| Design axis | Session affinity | Stateless server |
|---|---|---|
| Request routing | Later requests prefer or require one instance | Any eligible instance can handle the request |
| Working state | May remain in instance memory | Must arrive with the request or come from shared storage |
| Instance loss | Requires restoration, rebinding, or explicit failure | Usually permits another instance to continue |
| Coordination cost | Lower on the active path | More serialization, storage, or locking work |
| Main risk | Hidden dependence on one process | Incomplete or inconsistent state reconstruction |
Affinity is reasonable when active state is expensive to move or when one worker must coordinate a live operation. Stateless handling is preferable when horizontal failover and interchangeable workers matter more. A practical harness can combine both: use affinity for efficient execution while committing authoritative workflow state durably.
The Rifty take
We treat session affinity as an optimization with a failure obligation, not as a durability strategy. We accept the coordination cost of durable state at meaningful boundaries so that losing one worker does not erase operator control. If the harness cannot explain how a pinned session recovers, it should fail visibly rather than manufacture continuity.
Implementation checks
- Define exactly which state is instance-local and which state is authoritative.
- Keep the affinity key opaque, scoped, expiring, and separate from authorization credentials.
- Record binding creation, expiry, migration, and failed routing in the execution trace.
- Test worker loss before, during, and after a side-effecting tool call.
- Preserve approval decisions, execution limits, and stop conditions across rebinding.
- Use checkpoints, journals, and idempotency controls where retries could duplicate effects.
- Reject or reconcile a request when the previous outcome is unknown.
- Monitor per-instance session count, load, and repeated rebinding.
- Make affinity loss visible to the operator instead of silently starting an empty session.
Frequently asked questions
Does session affinity make an agent run durable?
No. Session affinity keeps requests near in-memory state, but that state can disappear with the owning instance. Durability requires authoritative state outside the process, such as checkpoints or an execution journal, plus a recovery procedure that distinguishes safe continuation from retry, reconciliation, or an unknown outcome.
When should an agent harness use session affinity?
Use it when one worker coordinates a live loop, holds costly working context, or manages a connection that cannot move cheaply between instances. Use it only when the reduced coordination cost justifies the coupling and the harness has explicit behavior for instance loss, session expiry, and rebinding.
Can an affinity token also authorize tool access?
It should not. An affinity token should identify routing state, while authorization should independently verify identity, delegated authority, permission scope, and approval conditions. Combining those roles makes routing metadata security-sensitive and can let a caller reach privileged session state without proving permission to act through it.
What should happen when the pinned instance fails?
The harness should follow a declared recovery contract: restore from durable state, resume from a checkpoint, reconcile an uncertain side effect, or stop with a visible failure. It should not route to an empty worker and pretend the session continued, because missing history can invalidate approvals, budgets, and tool outcomes.