Stateless Server

How it works

A stateless server treats every request as self-contained. It reads the request, does its work using only the inputs supplied plus shared backing stores it can look up, returns a response, and forgets everything local. No client-specific memory survives between calls inside the server's own process.

The pattern is a policy about where state lives, not a claim that state disappears:

1. The request carries or references everything needed to process it — identity, parameters, and a pointer to any prior context.
2. Durable state (sessions, history, work-in-progress) is externalized to a database, cache, queue, or object store.
3. Any server instance can handle any request, because none holds privileged local knowledge.
4. Scaling is horizontal: add instances behind a load balancer; a lost instance forfeits only its in-flight work.

Statelessness is a discipline you enforce, not a property you get for free. In-memory caches, sticky sessions, mutable module globals, and local file writes quietly reintroduce per-instance state. The test is simple: if two identical requests routed to two different instances can diverge, the server is not stateless.

Why it matters in an agent harness

An agent harness runs long, branching, failure-prone work: tool calls, model turns, retries, subagents. If the process that carries a run also is the run's memory, a restart erases the trajectory and a crash strands it.

A stateless request layer with externalized run state gives you the properties a harness depends on:

• Reversibility and resumption. When the agent's progress lives in a checkpoint store rather than a worker's heap, you can kill and respawn workers freely. A drain that dies mid-run re-queues and resumes from its last checkpoint.
• Observability. State written to an external store is state you can inspect. Hidden in-process state is state you cannot audit after the fact.
• Blast-radius control. A poisoned or wedged instance is disposable. You replace it instead of debugging it live, because it holds nothing unique.
• Reproducibility. Given the same externalized inputs, a replayed request behaves the same way. That is the substrate evals and regression checks stand on.

The harness lesson is directional: keep the compute layer stateless and push the state — run history, context, checkpoints — into durable, addressable stores. Statelessness at the edge is what makes clean recovery possible underneath.

Stateless Server vs Stateful Session

The distinction changes how you design recovery, so it is worth making explicit.

Stateful sessions are not wrong — they are fast and simple. They become dangerous when a long run depends on one process staying alive, which is exactly the shape of agent work.

The Rifty take

We treat statelessness as a control property, not a performance trick. The value is not throughput; it is that a worker holding no unique state is a worker you can kill without thinking. We push run state into checkpoints and treat every compute node as replaceable. The tradeoff we accept is the operational weight of an external state store and stricter request contracts — a price worth paying so recovery is boring and no long run ever hinges on one process surviving.

Common failure modes

• Accidental state. A local cache, sticky session, or module global that a later request reads. If two instances can disagree on identical inputs, statelessness is already broken.
• Confusing stateless with idempotent. A stateless server can still fire duplicate side effects on retry. You need idempotent handlers on top, or re-delivery double-writes.
• State store as new single point of failure. Externalizing state concentrates risk in the backend. Give it the durability, backups, and availability the whole system now inherits.
• Chatty context reloads. Rehydrating heavy context on every request adds latency. Cache reads where safe, but keep the authoritative copy external so no instance becomes irreplaceable.
• Partial writes without checkpoints. If a run's progress is only half-externalized, a mid-run crash leaves state you cannot cleanly resume from. Checkpoint at boundaries the harness can restart from.

Frequently asked questions

Is a stateless server actually memoryless?

No. A stateless server keeps no client-specific state in its own process between requests, but the system still holds state — in databases, caches, and checkpoint stores. Statelessness moves state out of the compute layer into durable, addressable backends, so any instance can serve any request without relying on local memory.

How is statelessness different from idempotency?

Statelessness is about where state lives; idempotency is about whether repeating a request is safe. A stateless server can still trigger duplicate side effects on retry. You want both: stateless compute so any instance can serve a call, and idempotent handlers so re-delivery does not double-charge or double-write.

Why does a stateless request layer help an agent harness recover?

Because the run's progress lives outside the worker. When compute holds no unique state, a crashed or killed worker loses only in-flight work; the trajectory sits in an external checkpoint store. You respawn a fresh instance, it reloads the last checkpoint, and the long run resumes instead of restarting.

What silently breaks statelessness?

In-memory caches, sticky sessions, mutable module globals, and local file writes that later reads depend on. Each reintroduces per-instance state. The tell is divergence: if two identical requests sent to two instances can return different results, some hidden local state exists and your recovery guarantees no longer hold.

Does stateless mean slower because of external lookups?

It can add latency, since context comes from a store rather than local memory. In a harness that tradeoff is usually worth it: you gain horizontal scale, disposable workers, and clean recovery. Cache reads where safe, but keep the durable copy external so no single instance becomes irreplaceable.