How we evaluate agents before they touch production

Why evals come first

An agent that works in a demo has answered one question: can it do the task when everything goes right? Production asks a different one. Can it do the task when the customer record is half empty, the payment API times out and the refund policy changed last Tuesday?

We stopped trusting demos in our first year. One support agent closed 94% of test tickets in a notebook, then refunded the wrong order on its third day live because two customers shared a name. Nothing in the demo set looked like that. Since then, every Kairon agent passes the same three gates before it acts on real data, and none of them involve someone saying “looks good to me.”

Build the suite from real work

A good eval suite is a sample of the work the agent will actually do, graded the way a careful colleague would grade it. Synthetic prompts are useful for coverage, but they miss the odd cases that cause incidents. We start from history.

Pick twenty, then two hundred

For a new agent we pull twenty real cases from the system it will work in: closed tickets, past invoices, merged pull requests. We pick them by hand to cover the common path, the expensive mistakes and the ones that confused humans. Twenty is enough to find out whether the agent understands the job. Once it passes, we grow the suite to roughly two hundred cases by sampling at random, so our own taste stops shaping the test.

Write rubrics a stranger could grade

Every case gets a rubric with three to six checks. Each check is a yes or no question about the outcome, not the wording. “Refunded the duplicate charge only” is a good check. “Responded helpfully” is not, because two reviewers will disagree about it. If a check needs judgment, we write down what a pass looks like and include one example of each.

Three gates before production

The suite runs at three points. Each gate uses more realistic conditions than the last, and an agent has to clear all three in order.

1. Offline evals. The full suite runs against recorded tool responses on every change to the prompt, the tools or the model. The pass bar is set per agent, usually 95% of checks.
2. Shadow mode. The agent sees live traffic and plans every action, but nothing executes. We compare its plan with what the human on shift did and review every disagreement.
3. Canary. The agent acts on 5% of real work with approvals turned on for anything that moves money or messages a customer. It stays there for at least a week.

What to steal for your team

Start with twenty real cases and a rubric a stranger could grade. Add shadow mode before anything executes, and keep approvals on for actions that move money. You can build the rest of the harness later.