How to test customer service AI before going live

By Bernardo Gontijo, founder of NeftPublished on Updated on 10 min read

Summary

To test customer service AI, evaluate two things separately: whether it resolved the request and whether it followed the required process, such as providing a reference number, asking for confirmation before changing data and handing over to a person when needed. In the CXBench benchmark, published in October 2026, 80% of runs in which AI agents achieved the correct outcome violated a process rule. Before releasing AI, build test cases with known answers and automated checks.

Why is a correct answer not enough in AI customer service?

Because in regulated customer service, the customer may receive the right solution through the wrong process: without a mandatory notice, without confirmation before a record is changed or with a promise the company cannot keep. An evaluation that only checks the final outcome approves these interactions as if they were correct.

This risk was measured by CXBench, a benchmark published on arXiv on October 19, 2026 by researchers led by Boxi Yu. The study tasked seven language models with serving simulated customers across 120 tasks in six regulated sectors: banking, telecommunications, insurance, healthcare, utilities (such as water and energy) and government benefits.

CXBench measureOutcome
Runs evaluated840
Achieved the correct request outcome69.3%
Achieved the correct outcome and followed the process13.8%
Correct outcomes with a process violation80.1%
Source: Yu et al., CXBench, arXiv 2610.25954, version 1 (October 2026). AI-simulated customers, not real interactions.

The gap appeared in all tested models and varied by sector. In banking, 94.3% of runs achieved the correct outcome, but only 5.7% got both outcome and process right. For government benefits, the figures were 84.3% versus 4.3%.

What is a false pass in AI customer service?

A false pass is an interaction that delivers the correct business outcome but fails a process requirement. The bill was disputed, the duplicate was issued or the record was updated, yet a step required by regulation or company policy was missing. Common examples:

  • No reference number provided: the request was resolved, but the customer left without a number to track it.
  • Confirmation out of order: the AI changed data before the customer confirmed what would be changed.
  • Missing mandatory notice: a deadline, right or condition required for that request type was not disclosed.
  • Unwarranted promise: the AI guaranteed a deadline, discount or outcome the company cannot ensure.
  • Handoff without context: the customer asked for a person and had to repeat everything to the agent.

The authors stress that a false pass in the benchmark indicates failure to comply with a rule encoded in the test, not a legal finding of misconduct. For service operators, the value lies in the method: giving outcome and process separate scores.

What process rules does Brazilian law require of customer service?

Brazilian Decree No. 13,135/2026 replaced Decree No. 11,034/2022. Check the providers covered by Article 1 and applicable sector regulation. Requirements that can become test cases include:

  • Reference number: enable request tracking through a numeric record or electronic procedure generating a reference number (Article 18).
  • Continuity: do not ask customers to repeat or restart requests; an exception to repetition requires a substantiated technical reason (Articles 6–7).
  • Human option: test access to a person under the regulation's conditions and hours, including electronic and telephone channels (Articles 3, 12 and 14).
  • Retention: history has a six-month minimum after resolution; telephone recordings and voice or text messages have a 90-day minimum after service (Articles 18–19). Also validate sector rules.

Each rule becomes a verifiable item in the AI test. Does the reference number appear in the record? Did the handoff deliver the summary to the agent without asking the customer to repeat the request? Was the option of speaking to a person offered?

The company remains responsible for what its AI says. In February 2024, in Moffatt v. Air Canada, British Columbia's Civil Resolution Tribunal in Canada ordered the airline to compensate a passenger who followed incorrect chatbot information about bereavement fares, rejecting the argument that the chatbot was responsible for itself. In Brazil, the Consumer Protection Code states that sufficiently precise information provided to consumers binds the supplier (Article 30).

How do you build tests when no benchmark exists for your customer service?

Start with the answer and only then generate the question. This is the technique Nathanael Weill, an AI engineer at Air Canada, presented at the AI Engineer conference in October 2026: when the expected answer is known, a program can check the outcome without relying on expert opinion every round.

Video: How to Test AI Agents When No Benchmark Exists (AI Engineer, YouTube, ). Talk in English, approximately 20 minutes long.

According to Weill, his team evaluated each agent version in a weekly meeting with experts who did not always agree. With tests generated from known answers, the team moved from one evaluation per week to dozens per day. In a supplier-contract project, known rules produced 275 tests with defined answers.

“If you cannot evaluate the agent, you cannot improve it.”

Nathanael Weill, Air Canada, in How to Test AI Agents When No Benchmark Exists (AI Engineer, October 2026, at 7:26). Paraphrased translation from the Portuguese article.

Adapted to customer service, the method becomes five types of test case:

  1. Known answer: choose a passage from the knowledge base, write the question it answers and check that the AI reaches the same information.
  2. Planted rule: for each mandatory rule, such as providing a reference number or requesting confirmation before changing data, create conversations that require it and verify in the record that it happened.
  3. Out of scope: requests about prohibited topics or matters the AI cannot resolve must end in a clear refusal or a handoff to a person.
  4. Invariance: change the customer's name, address or neighborhood without changing the request. The answer and treatment must not change.
  5. Handoff: the customer asks for a person mid-conversation. The test passes if the agent receives the summary and the customer does not need to repeat the request.

Weill also warns that adding complexity to an agent often breaks what already worked. The full test suite should therefore run after every change to instructions, the knowledge base or the model, not just on the cases that prompted the change.

What criteria should each test interaction assess?

Give each interaction an outcome score and a process score, preferring automated checks whenever the record allows. The table below is a starting point for regulated sectors:

CriterionTypeHow to verify
Resolved the requestOutcomeFinal system state, such as a duplicate bill issued or a ticket opened
Provided the reference numberProcessNumber present in the transcript and record
Requested confirmation before changing dataProcessEvent order in the record
Stayed within allowed topicsProcessComparison with the configured topic list
Did not promise what the company cannot guaranteeProcessHuman review of a sample
Offered a person when requested or out of scopeProcessHandoff event in the record
Did not ask the customer to repeat the requestProcessTranscript after the handoff
Example criteria. Actual requirements come from sector regulation, the contract and company policy.

What does the research change in day-to-day service?

What the source showsWhat changes in customer serviceLimitation
CXBench: 80.1% of correct outcomes violated process rulesEvaluate AI with two scores: outcome and processAI-simulated customers and rules from other countries
Air Canada: tests based on known answers speed up evaluationGenerate cases from the knowledge base and mandatory rulesProject report, with no published measurement of the gain
Brazilian Decree No. 13,135/2026: reference number, continuity and human optionEach rule becomes a verifiable test caseScope defined in Article 1; check sector regulation

Can I use AI to evaluate AI customer interactions?

Yes, but only for what a program cannot check, and always calibrated against a human-reviewed sample. Event order, presence of a reference number and handoffs are objective checks; tone, clarity and unwarranted promises require judgment.

In CXBench, evaluation combined code-based rules with a language model acting as judge. Of the 466 false passes, 329 were detected only by the AI judge, 38 only by rules and 99 by both. Even so, when the AI judge approved the process, its approval agreed with an expert's in 69.7% of cases.

Weill reported the same problem at Air Canada: noise from the AI evaluator was comparable to that of the agent being developed. The practical rule is to check with code first and use the AI judge only where indispensable.

How do you safely launch customer service AI?

Release gradually and keep a person in the loop until the numbers demonstrate quality. A common approach:

  1. Narrow scope: one high-volume request type, or the hours when the team is unavailable.
  2. Gated actions: start with simulated actions or human approval, and enable automatic execution only after tests pass.
  3. Versions: every instruction change creates a new version, tested before publication, with a quick rollback to the previous one.
  4. A path to a person: when the customer asks or the case goes out of scope, the interaction moves to the team with the same reference number and conversation summary.
  5. Weekly sample: review real interactions using the same outcome and process criteria, turning every discovered failure into a new test case.

This is how Neft's AI customer service is designed. The company defines allowed and prohibited topics; each action can be disabled, simulated, subject to human approval or automatic; and configurations have drafts, versions and publishing, with rollback. When a case goes out of scope, the customer moves to the team with the same reference number and conversation summary, while recording, transcription and reports stay in the same record.

Frequently asked questions

What is the difference between evaluating the outcome and the process of an AI interaction?
The outcome tells you whether the customer's request was resolved. The process tells you whether the AI followed mandatory steps along the way, such as providing a reference number, requesting confirmation and offering a person. An interaction is correct only when it passes both.
How many test cases are needed before launching customer service AI?
There is no universal number. For every request type, cover all mandatory rules, prohibited topics and handoff to a person; then add a case for each real failure found. In the Air Canada talk, one project's known rules produced 275 tests.
Is the company responsible for what the chatbot or AI tells the customer?
In practice, yes. In Moffatt v. Air Canada (2024), a Canadian tribunal rejected the argument that the chatbot was responsible for itself. In Brazil, Article 30 of the Consumer Protection Code states that sufficiently precise information provided to consumers binds the supplier. Confirm how this applies to your case with legal counsel.
Can I use AI to assess interactions handled by AI?
Yes, for criteria requiring judgment, such as tone and unwarranted promises, provided it is calibrated against a human-reviewed sample. Event order, reference numbers and handoffs should be checked with code.
Can AI handle service on its own in regulated sectors?
Automation may operate within a tested scope while preserving a human option. For providers covered by Brazilian Decree No. 13,135/2026, Articles 12 and 14 provide for human service from 8 a.m. to 8 p.m., seven days a week, subject to specific conditions and sector regulation. Do not use the repealed eight-hours-per-day rule as a current criterion.

Sources

Videos, studies and documents used in this article. Preprints are versions published by their authors before peer review. This article was researched and written with AI assistance; research and quotes checked on October 26, 2026. Customer service regulation references updated on November 16, 2026 against Decree No. 13,135/2026 on Brazil's official Planalto website.

  1. Video

    How to Test AI Agents When No Benchmark Exists (opens in a new tab)

    Nathanael Weill (Air Canada), AI Engineer channel · published on

    Testing agents from known answers, invariance testing and caution when using AI as judge.

  2. Research paper (preprint)

    CXBench: A Benchmark for Process-Compliant LLM Agents in Regulated Customer-Experience Workflows (opens in a new tab)

    Boxi Yu, Lionel Briand, Emir Muñoz and others (arXiv 2610.25954, version 1) · published on

    False-pass figures: 80.1% of correct outcomes violated process rules, with AI-simulated customers in six regulated sectors.

  3. Legislation

    Brazilian Decree No. 13,135 of October 9, 2026 (customer service regulation) (opens in a new tab)

    Office of the President of Brazil · published on

    Reference numbers, continuity, human option and retention; replaces Decree No. 11,034/2022.

  4. Court decision

    Moffatt v. Air Canada, 2024 BCCRT 149 (opens in a new tab)

    Civil Resolution Tribunal of British Columbia (Canada) · published on

    The airline was held responsible for incorrect information provided by its own website's chatbot.

  5. Legislation

    Brazilian Consumer Protection Code (Law No. 8,078/1990) (opens in a new tab)

    Office of the President of Brazil · published on

    Article 30: sufficiently precise information provided to consumers binds the supplier.

Want to serve your customers by video?

Take the free assessment or try Neft for 15 days, with no credit card.