How it works
An agent usually receives more than a tool name and input schema. It also receives descriptive text explaining when the tool should be used, what it returns, and how the model should behave around it. Tool poisoning turns that discovery surface into an instruction channel.
A poisoned description might tell the model to prefer one tool, include unrelated data in its arguments, conceal a step, or treat tool output as authoritative. The instruction can influence reasoning before any tool executes. That makes the attack easy to miss if monitoring begins only at invocation time.
The mechanism is a short control loop:
- The harness discovers or loads a tool and places its metadata in model context.
- The model interprets descriptive metadata alongside the operator’s request and system instructions.
- Adversarial text changes tool selection, argument construction, or subsequent reasoning.
- The harness validates the call weakly or treats model intent as sufficient authority.
- The tool acts within whatever permissions and credentials the harness supplied.
The description does not grant capability by itself. Damage occurs when untrusted discovery text can steer a model that holds consequential permissions. The security boundary therefore sits between model-generated intent and authorized execution, not merely between the model and the tool catalog.
Why it matters in an agent harness
Tool poisoning exposes a basic mismatch in many agent designs: text is used both to describe capabilities and to influence behavior. Models cannot reliably treat one natural-language passage as inert documentation while treating another as instruction. A harness should assume that any text placed in context can affect the trajectory.
The first engineering consequence is loss of control over tool choice. A model may select a tool because its description contains persuasive instructions rather than because the tool satisfies the task. Restricting the visible tool set reduces this selection surface, but it does not make the remaining descriptions trustworthy.
The second consequence is permission laundering. An operator may authorize a narrow task, while a poisoned tool induces the model to request additional files, credentials, recipients, or destinations. If the execution layer checks only whether the model emitted valid arguments, the tool can turn syntactic validity into unintended authority. Capability is not permission, and model selection is not authorization.
The third consequence is weak observability. A trace that records only the final call can make the action look reasonable. Investigation requires the exact tool description shown to the model, its source, its version or digest, the available alternatives, the generated arguments, and the policy decision that allowed execution. Without that context, operators cannot distinguish a poor model choice from a compromised discovery surface.
Reversibility also matters. Read-only retrieval may be recoverable, although disclosure cannot be undone. Sending messages, changing access, publishing content, or executing code can create a much larger blast radius. High-impact calls need deterministic checks, narrow credentials, approval gates where judgment is material, and compensating actions where reversal is possible.
Tool poisoning should also be part of evaluation. A harness can inject conflicting, coercive, or data-exfiltration instructions into test descriptions and verify that execution policy still rejects unauthorized effects. The useful result is not that the model ignored every malicious phrase. The stronger result is that model susceptibility could not cross the control boundary.
Tool poisoning vs prompt injection
Tool poisoning is a form of prompt injection distinguished by where the adversarial text enters and which trust decision it targets. That distinction changes where controls belong.
| Concern | Tool poisoning | Prompt injection |
|---|---|---|
| Entry point | Tool descriptions, schemas, annotations, or discovery metadata | Retrieved content, user input, messages, documents, or other contextual text |
| Primary target | Tool selection, arguments, and trust in tool behavior | The model’s broader instructions, reasoning, or output |
| Main control point | Tool registry, discovery pipeline, permission policy, and invocation gate | Context assembly, content isolation, instruction hierarchy, and action policy |
| Evidence to retain | Exact discovered metadata and the policy decision for each call | Exact injected content, its provenance, and its effect on the trajectory |
The categories can overlap. A poisoned description may direct the model to retrieve another malicious payload, while an injected document may steer the model toward a dangerous tool. The practical rule is to track provenance at every context boundary and authorize effects independently of the text that proposed them.
The Rifty take
We treat tool descriptions as untrusted behavioral input, even when they arrive through a valid protocol or approved registry. We optimize for narrow tool exposure and deterministic authorization at the moment of effect. We accept some friction around consequential actions because fluent model intent is not proof of operator consent.
Implementation checks
- Record the exact description, schema, source, and version or digest presented to the model.
- Allowlist tool sources and make catalog changes visible in review and audit records.
- Expose only the tools required for the current stage instead of a session-wide catalog.
- Validate argument semantics, destinations, data classes, and requested effects, not only schema shape.
- Bind credentials to the smallest practical resource, operation, and lifetime.
- Require approval for irreversible, externally visible, or authority-changing actions.
- Keep tool output untrusted when it returns instructions or requests additional access.
- Test poisoned descriptions that demand secrecy, preference, credential access, or unrelated actions.
- Fail closed when metadata provenance, authorization state, or the proposed effect is ambiguous.
- Ensure traces connect discovery text, model choice, policy evaluation, invocation, and result.
Frequently asked questions
Is tool poisoning possible if the tool itself is not malicious?
Yes. The executable tool can behave exactly as documented while compromised or untrusted discovery metadata steers the model toward an inappropriate call. The harness must evaluate both layers separately: whether the tool implementation is acceptable and whether this invocation is authorized for the operator’s present task.
Can stronger system prompts prevent tool poisoning?
No system prompt provides a sufficient execution boundary. It may reduce susceptibility, but the model still interprets tool descriptions as contextual text. Consequential effects need deterministic permission checks, narrow credentials, semantic argument validation, and approval gates that do not depend on the model correctly recognizing an attack.
What should an agent trace capture for a suspected poisoned tool?
Capture the exact tool metadata shown to the model, its provenance and version, the other available tools, relevant operator instructions, generated arguments, policy results, credentials or scopes used, tool output, and subsequent actions. This evidence connects the injected text to the execution trajectory without assuming causation from the final call alone.
How should tool poisoning be tested?
Insert adversarial instructions into test tool descriptions and observe both model behavior and enforcement outcomes. Cover requests for unrelated data, concealed actions, expanded permissions, preferred selection, and external side effects. The harness passes when unauthorized effects are blocked and visibly recorded, even if the model initially proposes them.