How it works
Risk-proportionate control starts by treating agent actions as different kinds of operational commitments. Reading a file, drafting a note, modifying a production record, spending money, deleting state, and using a privileged credential do not deserve the same control surface. The harness classifies the action before it executes, then applies controls that match the expected consequence if the action is wrong and the practical cost of undoing it.
A typical control loop looks like this:
- Classify the proposed action by write scope, external effect, credential sensitivity, financial cost, user visibility, and reversibility.
- Assign the action to a control lane: autonomous, logged, sandboxed, approval-gated, or blocked.
- Bind the lane to concrete machinery: scoped credentials, dry runs, diff review, checkpointing, execution limits, or human approval.
- Record the decision, inputs, actor, permission scope, and result in an operation trail.
- Recalibrate the policy when incidents, near misses, or routine operations show that the lane is too loose or too restrictive.
The important point is that risk is not model confidence. A model can sound confident while proposing a destructive action. A model can also be uncertain while doing harmless exploration. The harness should control the action's consequence, not the tone of the answer.
Why it matters in an agent harness
An agent harness exists because a capable model still needs bounded authority. Risk-proportionate control gives that authority shape. It keeps the system from collapsing into two weak defaults: approve everything because manual review is expensive, or review everything because autonomy feels unsafe.
The engineering outcome is more specific. Low-consequence actions can stay fast. Search, local summarization, planning, schema validation, and read-only inspection often need traceability more than interruption. The operator should be able to see what happened, but the harness should not force a meeting with every harmless step.
High-consequence actions need a different posture. Writing to production systems, using broad credentials, publishing externally, moving money, deleting state, changing permissions, or sending messages under a human identity should cross a harder boundary. The control can be an approval checkpoint, a dry-run artifact, a diff, a narrowed credential, a reversible transaction, or a hard block.
This matters because agent failures often become expensive through permission and timing. A bad paragraph is editable. A bad database migration, public post, credential leak, or destructive filesystem operation is a different class of event. The harness should encode that difference before execution rather than discover it during cleanup.
Risk-proportionate control also improves observability. If every event is logged with the same flat severity, the trail becomes noise. If the harness records why an action was allowed, gated, or blocked, operators can audit the policy itself. They can ask whether the system gave too much autonomy to a risky path or wasted attention on actions that were already reversible.
The tradeoff is policy work. Someone has to name the risk classes and keep them current. That is still cheaper than relying on broad permissions and heroic human vigilance after the fact.
Risk-proportionate control vs least privilege
Both ideas reduce harm, but they operate at different layers. Least privilege constrains what authority is available. Risk-proportionate control decides how much process surrounds a specific use of authority.
| Concept | Primary question | Harness implication |
|---|---|---|
| Least privilege | What is the smallest permission set this actor needs? | Scope credentials, tools, filesystem access, and API capabilities narrowly. |
| Risk-proportionate control | What control should surround this proposed action? | Choose logging, sandboxing, approval, rollback, or blocking based on consequence and reversibility. |
The distinction changes design decisions. A tool may be correctly scoped and still need approval for a particular operation. For example, an agent may have permission to publish content, but a public release under a brand name can still require a diff, provenance check, and approval gate. The credential is necessary. It is not the whole control system.
The reverse is also true. A low-risk action should not require broad permission just because it is easy to approve. Read-only inspection, local analysis, and draft generation can often run autonomously inside a narrow sandbox. Proportionate control is not a license to widen authority. It is a way to spend oversight where it changes the outcome.
The Rifty take
We optimize for controls that track consequence, not anxiety. A harness should make routine reversible work cheap, visible, and fast, while making irreversible or externally visible work slower by design.
The boundary we enforce is simple: autonomy is earned per action class, not granted to an agent personality. When the action can change durable state, spend trust, or escape the sandbox, the surrounding system has to carry more judgment than the model response does.
Implementation checks
- Define risk classes in operational terms: read-only, local write, reversible external write, irreversible external write, privileged change, and unknown outcome.
- Bind each class to explicit controls: log-only, sandbox, dry run, diff review, approval checkpoint, narrowed credential, rollback plan, or fail-closed block.
- Treat reversibility as a first-class input. A backed-up local edit is not the same risk as an external deletion with no recovery path.
- Separate model confidence from action risk. Do not lower controls because the agent presents a fluent rationale.
- Use scoped credentials and permission boundaries so a mistaken low-risk classification cannot silently access high-risk authority.
- Require durable traces for gated and privileged actions: proposal, approval, actor, credential scope, result, and recovery path.
- Review blocked and escalated actions periodically. Too many false escalations teach operators to ignore the control surface.
- Add regression tests for known incidents and near misses, especially where a harmless-looking tool call can trigger an external side effect.
Frequently asked questions
Is risk-proportionate control the same as human-in-the-loop review?
No. Human review is one possible control, not the whole pattern. Risk-proportionate control may choose logging, sandboxing, scoped credentials, dry runs, approval, or blocking. The point is to match the control to the action's consequence and reversibility, rather than inserting a person into every step.
What signals should determine the risk level of an agent action?
The strongest signals are consequence and reversibility. Check whether the action writes external state, uses sensitive credentials, affects users, spends money, changes permissions, deletes data, publishes publicly, or creates an unknown outcome. Model uncertainty can inform review, but it should not replace action-level risk classification.
How does this help an autonomous agent move faster?
It removes unnecessary friction from low-risk work while preserving hard boundaries around dangerous work. Read-only inspection, planning, and local reversible edits can proceed with traces instead of approvals. High-impact actions still slow down through checkpoints, narrowed permissions, or recovery requirements.
Where do teams usually get this wrong?
They often control by agent identity instead of action class. A trusted agent receives broad authority, or an untrusted agent is forced through constant review. Both are brittle. The harness should ask what this specific action can change, how it can be undone, and what evidence must survive afterward.
Can risk-proportionate control be fully deterministic?
Parts of it can be deterministic, especially tool scopes, permission classes, approval requirements, and fail-closed rules. The risk classification may still involve policy judgment for ambiguous actions. A strong harness makes those judgments explicit, records the reason, and avoids silent escalation or silent permission widening.