Data Locality

How it works

Data locality answers a single question for every byte an agent touches: does this stay on infrastructure I control, or does it leave? Treat it as a policy applied at each hop in the agent's data path, not a one-time storage choice.

Trace the path:

• Inputs — the documents, secrets, and context you load into the prompt.
• Model call — whether inference runs on your hardware, a private endpoint, or a shared hosted API.
• Tool calls — every tool, MCP server, or API the agent reaches, and what each one forwards.
• Outputs and logs — traces, embeddings, and cached context, which quietly persist data in new places.

At each hop you decide: keep it local, send a redacted subset, or route it out under a contract. The default an agent inherits is "everything flows to whatever the loop calls," so locality is the discipline of overriding that default on purpose. It is enforced with network egress rules, scoped credentials, redaction before the boundary, and provider agreements — not with intentions. The useful mental model is a set of concentric zones (local, private, external) with an explicit, logged gate between each.

Why it matters in an agent harness

An agent loop multiplies data movement. A single task can fan out into dozens of tool calls and model turns, each a potential exit for data you meant to keep in. Without a locality policy, your effective data boundary is the union of everything any tool might transmit — usually far wider than you assume.

Locality is what makes the rest of your controls meaningful:

• Blast radius. If a prompt injection or a buggy tool leaks context, locality decides whether the leak stays inside a sandbox or reaches a third party you cannot claw back.
• Reversibility. Data you kept local can be deleted. Data you sent to an external service is often unrecoverable — you cannot roll back a disclosure.
• Observability. When data stays on infrastructure you own, you can log and audit every access. Once it crosses a boundary, you inherit the provider's visibility, which is often none.
• Permissions. Least-privilege on tools only holds if those tools honor the same zones. A low-privilege tool that ships context to an external log breaks the boundary anyway.

The practical payoff: locality converts "we hope nothing sensitive escaped" into "sensitive data physically could not reach that endpoint."

Data locality vs data residency

The two are often conflated, and the difference changes design decisions.

Residency is a legal constraint about storage location. Locality is an engineering constraint about movement. You can satisfy residency (data stored in-region) and still violate your intent on locality (that same in-region data streamed through a shared model endpoint mid-task). Design for locality first; residency is a subset you then pin down.

The Rifty take

We treat every boundary crossing as a decision that gets logged, not a convenience that happens by default. We would rather run a smaller local model, or redact aggressively before a hosted call, than buy marginal answer quality by shipping raw context to an endpoint we cannot audit or revoke. The tradeoff we accept is more plumbing and occasionally weaker outputs; the boundary we enforce is that sensitive data never leaves a zone without an explicit, recorded gate.

Common failure modes

• Silent egress through tools. A tool or MCP server forwards full context to a hosted API you never vetted. Read what each tool transmits, not just what it returns.
• Logs and traces as leaks. Observability pipelines and prompt caches persist sensitive context in external systems — the same data you carefully kept away from the model.
• Embeddings count. Sending text to a hosted embedding endpoint is egress, and a vector store is a copy of your data in another form.
• Locality assumed, not enforced. "It runs on our servers" without egress rules means any tool can still reach the internet. Enforce with network policy, not documentation.
• Redaction after the boundary. Stripping secrets in post-processing is too late. Redact before the data leaves the zone, or it has already left.

Frequently asked questions

What's the difference between data locality and data residency?

Residency is a legal question about which jurisdiction stores your data at rest; locality is an engineering question about where data actually moves at runtime. You can meet residency in-region and still break locality by streaming that data through a shared external endpoint. Solve locality first, then pin residency as a subset.

How do I keep sensitive data local when the agent calls a hosted model?

Redact or tokenize before the boundary, not after — once raw context leaves, it is gone. Send the minimum the model needs, keep secrets and identifiers in local tools, and enforce egress rules so only approved endpoints are reachable. For the most sensitive paths, route to a local or private-endpoint model.

Does adding an MCP server change my data locality?

Yes. An MCP server is another hop in the data path, and it can forward whatever context you pass to a hosted backend you never vetted. Check what each server transmits and where, not just what it returns. Treat every tool and transport as a potential egress point governed by the same zone policy.

Do embeddings and logs affect data locality?

They do, and they are the most-missed leaks. A hosted embedding call is egress, and the resulting vector store is a copy of your data in another form. Traces, prompt caches, and observability pipelines persist sensitive context in external systems — the same data you kept out of the model quietly reappears there.

Is running on-prem enough to guarantee locality?

No. "It runs on our servers" without network egress rules means any tool in the loop can still reach the public internet and ship data out. Locality is enforced by policy — scoped credentials, egress allowlists, and pre-boundary redaction — not by where the process happens to run.