Continuous EvalOps for AI Agents: What to Rerun After Changes

Build a baseline you can repeat

Start with one important workflow and a set of representative cases. Record the customer request, allowed actions, approved information sources, and expected outcome. Include ordinary requests, ambiguous requests, and situations where the correct answer is to stop or ask a person. Keep previous failures in the set so a repaired problem does not quietly return.

Store the test cases with their scoring criteria and the configuration used for each run. Useful configuration details include the model, prompt version, tool definitions, knowledge-source version, and relevant application release. Without that record, a change in results is hard to explain and a previous pass is hard to reproduce.

OpenAI's evaluation guidance recommends task-specific evaluations, production-representative data, human calibration, and continuous evaluation. A practical implementation is a stable baseline plus new cases from observed failures. It should grow deliberately, with coverage tied to the work your agent actually performs.

Separate release tests, monitoring, and reruns

A repeatable suite exercises known scenarios in a controlled environment. Run it before releasing relevant changes and compare the candidate with the approved baseline. It gives you direct evidence about those cases, including difficult situations that might rarely appear in normal traffic.

Sampled monitoring reviews a defined portion of operational activity. It can reveal new language, changed customer needs, or failures missing from the suite. A sample does not cover every interaction, and quiet monitoring does not prove that every action was correct. State the sampling method, review window, exclusions, and escalation owner.

A scoped rerun repeats the tests affected by a particular change or fix. It answers a narrower question and should be labeled accordingly. Continuous EvalOps can combine all three, but an occasional rerun is not round-the-clock surveillance. Agree the cadence and coverage explicitly instead of relying on the word “continuous.”

Match the rerun to the change

• Prompt change: Repeat the target behavior plus nearby refusal, approval, and handoff cases. A shorter instruction can unintentionally remove a condition.
• Model change: Compare the broader workflow suite, including tool selection and repeated runs where outcomes vary.
• Knowledge update: Test revised facts, conflicts with older documents, and what happens when the answer is absent.
• Tool change: Verify inputs, results, permission checks, timeouts, and recovery for every affected connection.
• Offer or checkout change: Recheck price presentation, final totals, approval boundaries, and payment or order states.
• Permission change: Exercise both allowed and denied actions for affected roles, including direct attempts outside the normal conversation.

These are starting rules for selecting coverage, not universal release criteria. A change shared across multiple workflows needs wider testing than a corrected sentence in one reference document. Record why a subset was sufficient, and broaden the run when failures suggest effects beyond that subset.

A concrete example: updating a service policy

Imagine a fictional repair business changes its cancellation window from two days to one day. Updating the policy document is only the first step. The agent might retrieve an older FAQ, keep a previous conversation's answer, or apply the right policy while sending the wrong message to staff.

A scoped rerun should ask about cancellation inside and outside the new window, refer to the old policy explicitly, and request an exception. Check the answer against the current policy and inspect any downstream action. Add a human review for an ambiguous customer request, such as “I may not make it tomorrow,” where immediately canceling would be inappropriate.

If the policy update also changes the cancellation tool, expand coverage to authorization, failed requests, and retries. The report should distinguish a corrected answer from a verified cancellation workflow. Those findings support different release decisions.

Keep people involved where judgment matters

Automated checks are useful for exact values, required fields, forbidden actions, and known outcomes. Human reviewers should assess material omissions, inappropriate promises, and whether an escalation gives staff enough context. Use a shared rubric and resolve disagreements before treating a subjective score as a release gate.

Have reviewers examine a sample of passes as well as obvious failures. Otherwise, the review process only sees mistakes its existing checks already recognize. Limit access to customer information, remove unnecessary identifiers from evaluation examples, and agree retention rules for stored traces before collecting them.

Make each report support a decision

A useful recurring report names the evaluated version, coverage, failures, changes since the baseline, and unresolved risks. Give each material issue an owner and a retest condition. Define when to pause an action, keep a workflow in draft mode, or revert a release. Keep serious permission failures visible even when the overall pass rate improves.

Our AI Quality & Agent Operations service connects this review cycle to an initial baseline. Start with a small business AI evaluation checklist, then agree which recurring checks your workflow needs. The objective is a defensible operating decision supported by current evidence.