What is evals in AI? A definition and which to build first

Key takeaways

• An eval is one thing: an input, grading logic on the output, and a success measure.
• Defining it is the easy part. Deciding which eval to build first is the real cost.
• Start with a cheap, exact check on the failure that already burned you.

What an eval actually is

If you searched what is evals in AI, you have probably already read the definition four times. Every top result agrees, so let me collapse them into one line you can keep.

At its core, an eval is a test for an AI system: you give the AI an input, apply grading logic to its output, and measure success against a criterion you set. That is the whole shape. Input, grader, measure.

The vendor docs say the same thing in their own words. OpenAI's guidance frames evals as tests that check model outputs against style and content criteria you specify. A widely-read practitioner writeup calls an LLM eval the systematic measurement of a pipeline's quality that goes beyond a single number. Ask the same question in practitioner forums and you get the same answer: assess how well the system performs the task, decides, and interacts. So if you have seen "what is AI evals" answered a dozen ways, relax. It is one concept wearing different clothes.

Here is the part the clean definition hides. "A test for an AI system" sounds simple, and it is not. The difficulty does not live in the input or the pass/fail line. It lives in the grading logic. The rest of this page is about that.

Why evals matter: you find the failures before your users do

Ship an AI feature without evals and you still find every failure. You just find them last, in production, with your users watching.

The failure modes are familiar. A chatbot states a fact that is not true. A code generator sails through the demo and falls over on an edge case. A retrieval system pulls the wrong context and answers confidently from it. None of these show up when you click through happy paths by hand. They show up at scale, in the inputs you did not think to try.

Evals move that discovery earlier. They are the shift-left move: catch the regression in a test run, not in a support ticket. And they exist because output quality is genuinely hard to enforce by hand once volume climbs. You cannot read every response. A human review queue does not scale, and it gets slower and less consistent exactly when traffic grows.

Be honest about what an eval does not do. It does not prevent failures. It makes them visible before your users hit them, and it makes a regression loud instead of silent. That is the trade you are buying.

The four kinds of eval, and where the unit-test analogy breaks

Most explainers reach for the same comparison. Evals are methods for measuring whether an AI feature performs well, analogous to unit and integration testing. It is a good on-ramp. If you know how to wire a test suite into CI, you already know where evals plug in.

There are roughly four kinds you will build.

• Assertion, or unit-style. A deterministic check on the output. Does it contain the required field. Does it match an exact string. Does it avoid a forbidden phrase.
• LLM-as-judge. A second model grades the output against a rubric, for things no exact rule captures, like tone or helpfulness.
• Trajectory, or agent. For a multi-step agent, you grade the path: the decisions, the tool calls, the recovery, not just the final answer. This is what agent evaluation means when people say it, assessing how well the agent performs tasks, decides, and interacts.
• Regression. A frozen case you never want to break again, run on every change. Evals also live as reusable registries you run repeatedly across models, which is what makes regression practical.

Now the part the analogy skips. A unit test asserts a deterministic equality. Two plus two is four, every run, forever. An LLM output varies run to run for the same input. So the moment you leave exact-match territory, the thing you have to trust is no longer the output. It is the grader. Your LLM-as-judge has its own quality problem, and now you have two systems to trust instead of one. The analogy holds beautifully for assertions. It breaks precisely where non-determinism and subjective grading enter, and for agent work, that is most of the surface.

Which eval to build first in a harness

This is the question the definitions never answer. You know what an eval is. You have a system half-built. Which one do you write first? Since evals are analogous to unit and integration testing, borrow the instinct a test suite already trains: start where a failure hurt, and freeze the case so it never comes back.

Here is the order I use.

The default is opinionated: start with the cheapest deterministic assertion on the failure that already cost you something. Not a broad quality suite. One exact check on one real bug. It runs in milliseconds, it never flakes, and it pays for itself the first time it stops a regression. The regression row is nearly as cheap, because a frozen case run through a reusable registry you run repeatedly across models keeps guarding every later change for almost no upkeep.

This is a judgment call, not a law. A retrieval-heavy system can invert the order, because your worst failures are subjective relevance problems that an assertion cannot see, so you reach for a judge sooner. Read your own failure history and let it pick the first row. If you are designing the wider control layer, evals are one lever among several in an agent harness, and the sequencing matters as much as the choice.

A worked example: one input, one grader, one pass/fail

Definitions get slippery until you walk one all the way through. Here is a small illustration, not a measured result from a real system, and not a rifty benchmark. Treat the shape as the point and the numbers as a stand-in.

Say your agent handles password resets. The eval has three parts.

• Input: the message "I forgot my password, help."
• Grading logic: check the output for two things. It must contain your real reset URL, and it must not invent a phone number or a fake support address.
• Success measure: pass if both conditions hold. Run it across twenty phrasings of the same request, and the eval passes only when every case passes.

That is the entire mechanism, and it is exactly the criteria-check shape the docs describe. Notice what makes it cheap: the grader is deterministic. There is no second model, no rubric, no subjective call. This is the "start here" row made concrete, and it is the honest answer to how to create evals for AI. You do not begin with a grand quality framework. You begin with one input, one check, one pass line, then you grow it. One caution before you lean on it: these numbers are invented for the walk-through, so the shape transfers but the pass line does not. Swap in your real reset URL and your own phrasings, and do not quote any figure here as a result, because none of it was measured.

AI eval tools, by what they're for

Search "AI eval tools" and you get ranked lists that mostly sell you the author's product. A more useful map sorts them by the job, because the categories matter more than the logos.

• Open-source metric frameworks. They hand you scorers and metrics; you wire them into your own code. Good when you want control and do not need a UI.
• RAG-focused tools. Built to score retrieval quality specifically, the "did we fetch the right context" problem.
• Agent observability. Built to watch traces of multi-step agents running in production, where the path is the thing you are grading.
• Full platforms. These automatically score production traces, convert failures into permanent regression tests, and gate CI/CD so a bad change cannot deploy. The regression-conversion loop is the part worth paying for, because it turns each incident into a test that stays fixed.

There is also a whole class of platforms built to evaluate and compare AI systems at scale, which is where evals grow from a single test into benchmark-style comparison run across many systems.

Say the quiet part plainly: most of the named tools in any given category are close substitutes. The "best eval tool in 2026" framing is mostly marketing. Choose by the eval type you actually need first, the assertion or the judge or the trajectory grader, and the shortlist picks itself. The logo is the last decision, not the first.

Where evals mislead you

A green eval suite feels like safety. Sometimes it is lying to you, and this is the part the definitional guides leave out.

The sharpest example is the LLM-as-judge that rewards the wrong thing. Point a judge at "helpfulness" and it will quietly start scoring longer answers higher, because verbose reads as thorough. Your score climbs week over week. The real task quality flatlines, or drops. You are now optimizing the grader, not the product. This is the concrete version of the point that a good eval goes beyond a single number: a single number is exactly what gets gamed.

Three failure modes to watch. Grader drift, where the judge's standard moves and you do not notice. Overfit metrics, where you tune the system to the eval instead of the task. Coverage gaps, where the suite is green because it never tests the input that breaks you.

The fix is not free, and I will not pretend it is. Output quality stays hard to enforce at scale without human review, so you spot-check the grader against human judgment and keep a small human-reviewed slice of traffic as ground truth. Automated scoring reduces that review; it does not replace it. That is engineering judgment, not a number from a published benchmark. But a suite nobody audits is a suite that tells you what you want to hear.

How to learn evals, and what "evals in Claude Code" means

The fastest way to learn evals is to write one, not to read a fifth explainer. Take the last bug you shipped. Write a single regression eval that would have caught it, then fold it into the reusable registry you run on every change and generalize outward from there. You will learn more from that one honest case than from any framework, and it seeds the regression suite you needed anyway. The practitioner threads on this land in the same place: start small, from a real failure, and grow.

On coding agents specifically. When people talk about evals for a coding agent, the idea is straightforward: the eval grades whether the agent's change actually did the task. Did the diff pass the tests. Did it fix the bug without breaking three others. That is a trajectory eval wearing a developer hat. "Evals in Claude Code" is not a formal, documented product feature, and no authoritative source in reach describes it as one, so read the phrase as this general coding-agent-eval idea rather than a specific capability you configure. If your tool ships something branded that way, check its own docs before you assume behavior; the concept is real and portable across tools, but the branded framing is shorthand, not a spec.

If you are building agents that have to survive production rather than pass a demo, evals sit inside a larger question of control. That is the discipline behind building agentic AI systems that hold together, and the reason autonomous agents need a harness around them before they touch anything that matters. Start with one eval on one real failure, and let the suite earn its next test.

Read the agent harness deep-dive to see where evals sit among the levers that decide what actually ships.