Systematic measurement of whether agents produce the right outputs at the right quality level.
Performance evaluation is the discipline of knowing whether your agent system is actually doing what it is supposed to do — not just that it is running without errors. It distinguishes between system health (is it up?) and output quality (is it good?).
An evaluation system requires three components. Ground truth: a set of reference inputs and their correct outputs, validated by domain experts. Metrics: quantitative measures that map model outputs to business outcomes, not just model-level statistics. Cadence: a regular schedule for running evaluations and reviewing results, with defined response actions for different outcome levels. The most important design principle is that metrics must measure what the business actually cares about. A customer service agent measured only on response time will optimise for speed at the cost of accuracy. Evaluation sets must also be kept current: a test set built from 2023 data becomes progressively less representative of 2025 production inputs.
A legal services firm evaluates its contract review agent on three metrics: recall (what percentage of issues identified by human lawyers does the agent also identify?), precision (what percentage of the agent's flagged issues are confirmed as genuine issues by human review?), and time-to-review (how long does it take the agent to process a 50-page contract?). These are measured weekly on a held-out test set of 30 contracts reviewed by human lawyers. When recall drops below 85%, the team investigates the miss patterns. When precision drops below 70%, the team investigates false positive causes. Results are published to the legal team monthly.
Without systematic evaluation, you don't know when your agent starts to fail. Model performance degrades gradually with distribution shifts in input data. Without measurement, no single interaction is bad enough to trigger an alert — but the cumulative degradation over months produces a system that no longer performs as originally demonstrated. Evaluation is how you maintain the performance you proved during deployment.
How this pattern fails in practice — and what to watch for.
The agent is optimised for the measured metric — response time, say — at the cost of output quality. It begins producing faster responses that are less accurate. Since accuracy is not in the metric, the evaluation system reports improving performance while the business outcome is deteriorating.
The evaluation set was built at deployment. Two years later, it no longer reflects the distribution of production inputs. New topics, formats, and user types have emerged that the test set doesn't cover. Evaluation scores remain high on the stale test set while real-world performance has degraded on the new input types.
The alert is set to trigger when performance drops below 70%. Performance has been declining from 92% to 75% over 18 months — always above the threshold. By the time it crosses 70%, the degradation has been visible in the data for over a year but no action was taken.
Seven things to verify before deploying this pattern in production.
Performance evaluation is central to the AIMA certification and appears in AIDA under D4. The exams test candidates on evaluation design: given a business scenario, what metrics would you define, and what would constitute an adequate evaluation cadence? CAIG examines the governance of evaluation: who reviews results, what decisions do they trigger, and how is this documented? CAIAUD auditors are examined on their ability to assess the adequacy of an organisation's evaluation programme.
The AIDA certification covers all 21 agentic design patterns with a focus on deployment safety, governance, and the PSF. Free to attempt.