AI agent evaluation.

Start with a golden set, not a framework

The first artifact is a file of real cases: the input, the context available at the time, and what a competent human considers a correct answer. Fifty cases beat zero, and fifty real cases beat five hundred synthetic ones. Pull them from the actual failure log, because that is where the distribution lives.

Version the golden set alongside the code. When someone argues about whether quality moved, the argument should be settled by a diff.

Score the things that have a right answer differently

• Deterministic checks. Did it call the right tool, produce valid JSON, respect the schema, stay under the token budget, refuse what it should refuse? These are assertions, not judgments — write them as tests.
• Grounding checks. Is every factual claim traceable to a retrieved, permitted source? A grounding rate is measurable without a human reading every answer.
• Rubric scoring. For open-ended output, define three to five criteria with explicit anchors, then score with a model and audit a sample by hand. A rubric with anchors is reproducible; a one-to-five "quality" score is not.

Offline and online are both required

Offline evals run against fixed cases and tell you whether a change is safe. Online evals run against live traffic and tell you whether reality matches your fixed cases. Offline alone drifts; online alone cannot gate a release. The pair is what lets you swap a model on a Tuesday.

Put the gate in CI

The eval suite earns its keep the day it blocks a merge. A workable gate has three thresholds: no deterministic check may regress, grounding rate may not fall below its floor, and rubric scores may not drop more than a stated margin. Anything else is advisory. Keep the suite fast enough to run on every pull request, and keep a longer nightly suite for the expensive cases.

The metrics that survive a model swap

• Task success rate on the golden set, per task type rather than in aggregate.
• Grounding rate — the share of claims with a retrievable source.
• Cost per correct answer, which is the only cost number that cannot be gamed by a cheaper wrong answer.
• Latency at the ninety-fifth percentile, because averages hide the experience people remember.
• Escalation rate — how often a human has to take over. This is the number executives actually feel.

Error budgets, borrowed from reliability engineering

Perfect accuracy is not a target, it is a way to never ship. Decide what an acceptable error rate is for each task given the cost of being wrong, publish it, and spend the budget deliberately. A workflow that files a ticket can tolerate more error than one that issues a refund. Once the budget is explicit, arguments about model choice stop being aesthetic.

Common failure modes

• Evaluating the model instead of the system, so retrieval and memory regressions go unnoticed.
• Golden sets built from happy paths, which pass forever and predict nothing.
• A model grading its own output with no human audit sample.
• One aggregate score, which hides the task type that is quietly failing.
• Evals that live in a notebook rather than in CI, and therefore stop running by week three.