Evals Are the Tests You Should Have Written First

Someone on your team changed a prompt last Friday. Nobody can say whether the agent got better or worse on Monday. Someone read a few outputs, they looked fine, and the change shipped.

For every other part of your codebase, the question has a boring answer: run the tests. For the LLM-powered part, often the part that touches customers directly, there is silence.

We have a name for this in ordinary software. It is called not having tests. We would not accept it from a payments integration or a migration script. But rename the component "AI" and the discipline quietly evaporates.

The industry has since invented a word for the fix: evals. The word is useful. The implication it carries is not, because it suggests a new practice, a specialist artifact, something you adopt later, when the AI part matures. Evals are the tests you should have written first. The renaming did not create a new discipline. It created permission to skip an old one.

I am not writing this from a distance. I run coding agents daily, and I have shipped prompt changes on nothing more than reading three outputs. I know the silence firsthand. That is what makes it worth naming.

Why the tests never got written

It is worth being fair about how this happened, because the reasons sound reasonable and they are still circulating.

The first reason is that the feature was born in a demo. A model call that produces a plausible paragraph looks finished in a way that a function returning null does not. Deterministic code announces its own incompleteness. Probabilistic output hides it. So the feature shipped at the moment it looked done, which is a different moment from when it was done.

The second reason is the non-determinism excuse: you cannot write a test when the output is different every time. This one deserves to be retired, because it misunderstands what tests assert. Good tests rarely assert exact output. They assert properties. The response cites a source that exists. The extracted total matches the invoice. The refusal triggers on the eleven things it must refuse on. None of these require determinism. They require you to say what "correct" means, which is the work the demo let you defer.

Engineers already know how to test non-deterministic systems. We do it for flaky networks, for load, for canary deploys. We assert on distributions and thresholds instead of exact values. Nothing about a language model makes that toolkit obsolete. What the model changes is the definition of a passing run, not the obligation to define one.

What actually changes

Honesty requires naming what is genuinely different, because three things are.

First, pass/fail becomes pass rate. A single green run of a probabilistic component tells you almost nothing; the same input can fail on the next call. So an eval runs a case many times and asserts a threshold, and the threshold is a business decision, not a technical one. Ninety-five percent sounds strong until you compound it: an agent whose every step succeeds 95% of the time completes a twenty-step workflow about a third of the time. Someone has to decide what rate is acceptable, per failure mode, knowing what a failure costs. That decision is the real deliverable. The test suite is just where it gets recorded.

Second, the suite decays. A conventional test suite is a snapshot: the code changes, you update the tests, truth holds still between commits. An eval suite is a subscription. The model provider ships an update you did not ask for. Your users drift into inputs nobody imagined. The prices, policies, and product names your prompts mention change quietly underneath them. An eval set written once and trusted forever converges on theater: green, comprehensive-looking, and asserting things that no longer matter. The maintenance is not overhead. It is the mechanism by which the suite keeps referring to reality.

Third, and this is the strongest objection to the framing of this essay: for open-ended output, deciding what counts as correct is itself new work. A generated summary has no invoice to check against. Teams solve this with model judges grading model output, and an uncalibrated judge is the original problem wearing a lab coat. Judges earn trust the same way the rest of the suite does: checked against human review until they agree often enough to be load-bearing. That calibration work is real and classical testing has no equivalent for it. But notice what it still is. A definition of correctness, written down late.

None of these differences argue for skipping evals. They argue for treating them as an operating practice rather than a one-time artifact, which is what testing was always supposed to be.

Writing the tests backward

If the agent is already in production without evals, there is a consolation: you are holding better test material than any team starting fresh.

Your production traces are a record of every way the system has actually failed. Most teams have the traces and have never grouped them. Do that. Pull the runs, sort the failures by mode, and turn each real failure into an eval case. If your logging cannot reconstruct why a run failed, that is finding zero, and it comes before anything else on this list.

Twenty cases mined from your own traces will catch more regressions than ten thousand rows of a generic benchmark, because they encode the specific ways your system meets your users, which is the only distribution that matters.

This is also the honest sequence. Evals invented in a vacuum encode what the team fears. Evals mined from traces encode what happens. The gap between those two lists is usually the most informative document the team has ever produced. In the LangChain survey of roughly 1,300 practitioners, quality and reliability was the most-cited barrier to shipping agents, and only about half reported having evaluation infrastructure at all. The barrier and the missing practice are the same item.

The cost is the point

Building evals slows the roadmap. Maintaining them slows it more. The threshold arguments alone can eat a sprint, and every week spent mining traces is a week not spent on features.

This is the same trade software settled once already. The old line about brakes applies: they are not there to slow the car down, they are the reason it can be driven fast. Skipping evals does not remove the testing from the system. It relocates it: to production, unstructured, run by your customers, with results reported to your support channel instead of your CI.

The prompt change last Friday either regressed something or it did not. That fact exists whether or not you have machinery to observe it. Evals are the decision to find out before someone else does.

brimtech works with engineering teams running AI inside systems they intend to keep. If your agent is in production and this essay describes it, the Agent Production Audit is where we usually start: two weeks, your real traces, a measured baseline, and a ranked fix list.