Eval

How it works

An eval is a fixture plus a grader. You freeze an input, run the system against it, and score the result with logic you control. The loop is deliberately mechanical:

• Fix the input. A prompt, a dataset row, or a recorded scenario the system must handle the same way every time.
• Run the system under test. The model, chain, or agent produces an output — or, for an agent, a full trajectory of tool calls and steps.
• Apply a grader. An exact match, a schema check, a deterministic verifier, or a model-based judge scores the output against your criterion.
• Aggregate. Pass rates, scores, and newly-failing cases roll up across the whole set.

The grader is where the design lives. Deterministic checks — did it return valid JSON, did the test suite pass, did the tool get called with the right arguments — are cheap and trustworthy. Model-based checks cover things you cannot express in code, like faithfulness to a source, at the cost of their own noise. The criterion itself is a decision you own, not a property of the model. An eval is only as honest as the bar you set and the set behind it.

Why it matters in an agent harness

Agents are stochastic and multi-step, so "it worked once" tells you nothing durable. An eval is how you convert that uncertainty into a signal you can gate on. Inside a harness, evals do three concrete jobs. They make a change reviewable: a prompt tweak or model swap either holds the pass rate or it does not, and you see which cases moved. They make behavior legible: a trajectory eval shows how the agent reached an answer, not just the final string, so you can catch a right answer produced by the wrong path. And they set the boundary for autonomy: the region where evals are green is the region where you are willing to let the loop run unattended. Everything outside it stays behind a human or a stricter guardrail. Without evals, "the agent is reliable" is a vibe. With them, it is a number attached to a specific set of inputs — one you can defend when it breaks.

Eval vs benchmark

The two get conflated, and the conflation costs you. A benchmark is a shared, public set that ranks systems against each other. An eval is a private set that answers whether your change is safe to ship. The distinction decides whether you build your own.

A high benchmark score is a claim about a model. A green eval is a claim about your system doing your job. You need both, but only one of them protects a deploy.

The Rifty take

We treat evals as the load-bearing wall of a harness, not a report card printed after the fact. We optimize for a small, adversarial set drawn from real failures over a large, comfortable one that mostly passes — the cases that hurt are the ones worth grading. We accept that a good eval set is slow and expensive to build, because the alternative is shipping on faith. The bar is ours to set, and setting it low to feel productive is the failure we watch for hardest.

Common failure modes

• Grading the easy cases. A set full of inputs the system already handles inflates the pass rate and hides the failures that matter.
• Overfitting to the eval. Tuning prompts until the set goes green teaches the system to pass the test, not to do the job.
• Trusting a noisy judge. Model-based graders drift and disagree; if you never spot-check the judge, you are measuring its mood.
• Scoring outputs, ignoring trajectories. A correct final answer reached through a forbidden tool call or a lucky guess is a failure you graded as a pass.
• Letting the set rot. Real inputs shift; an eval frozen at launch slowly stops describing the system you actually run.
• No regression gate. If a green eval does not block a red deploy, it is documentation, not a control.

Frequently asked questions

What is an eval in AI?

An eval is a repeatable test that supplies a fixed input to an AI system, applies grading logic to the output, and scores it against a criterion you set. It turns a subjective sense that the system 'works' into a measurable pass rate you can compare across changes.

How is an eval different from a benchmark?

A benchmark is a shared public set that ranks systems against each other; an eval is a private set that answers whether your specific change is safe to ship. A high benchmark score describes a model, while a green eval describes your system doing your job.

When should a grader be deterministic versus model-based?

Use a deterministic grader whenever the criterion is expressible in code — valid JSON, a passing test, a correct tool argument — because it is cheap and trustworthy. Reserve model-based judges for qualities like faithfulness or tone that resist exact matching, and spot-check the judge for drift.

Why do evals matter for agent autonomy?

The region where your evals stay green is the region where you can let an agent run unattended. Evals convert a stochastic, multi-step loop into a gate-able signal, so autonomy expands only as fast as your measured reliability, and everything outside that region stays behind a human or a guardrail.

What makes an eval set go stale?

Real inputs shift while a frozen set does not, so an eval built at launch slowly stops describing the system you actually run. Sets also rot when they overweight easy passing cases; the fix is to keep feeding in real failures so the set stays adversarial.