Key Takeaways

• Traditional QA frameworks evaluate what an agent said but effective AI agent evaluation must assess what the agent did, including tool calls, decision paths, and outcomes.
• Tool call failures are among the most dangerous silent errors in production: the conversation sounds normal, but the underlying action was wrong, incomplete, or never processed, and no transcript will reveal it.
• Guardrail breaches are a real compliance risk in regulated industries; without a systematic detection layer running on every conversation, violations only surface after a customer complaint or audit.
• Goal failures are chronically undercounted; containment metrics log interactions as "handled" even when the intended outcome was never actually delivered to the customer.
• Latency degradation quietly erodes CX before any dashboard flags it; as one contact center director discovered, the first signal was a drop in inbound volume, not a monitoring alert. Learn how AI-powered contact centers are reducing latency by engineering their AI agent the right way.
• A complete AI agent evaluation strategy requires an evaluation layer embedded within the agent's own system, one with access to the full decision log, not just the transcript an external tool can see.

Introduction

After building and deploying AI agents across enterprise contact centers and sitting in hundreds of product conversations, I have a clear picture of where production failures actually originate. Pre-deployment AI agent evaluation focuses almost entirely on language quality: comprehension, accuracy, tone, handling of edge-case phrasing. These are real concerns and worth solving. Production failures that drive repeat contacts and compliance exposure trace back to four different categories, none of which a transcript-level evaluation is built to detect. Below are the four failure types:

1. Tool call failures

Enterprise AI agents do not just answer questions. They take actions: reading from CRM records, writing back to ticketing systems, verifying customer identity, processing requests against backend integrations. Each of those actions is triggered by an instruction the agent sends to an external system. The agent decides which system to contact, assembles the instruction, and sends it.

These instructions fail in ways that produce no visible surface signal. The agent contacts the wrong system entirely. It contacts the correct system but assembles the instruction incorrectly, so the data it receives back is wrong or the action it triggers is not the one intended. It skips a required step and moves forward treating the action as complete. The conversation can feel entirely normal to the customer throughout. The call ends, it is counted as handled, and the underlying action was wrong, incomplete, or never processed. The customer finds out when they check their account two days later.

2. Guardrail failures

Every AI agent deployment comes with a set of rules about what the agent can and cannot do: which actions it is authorized to take, which topics are outside its scope, when it should hand off to a human. These rules fail in two ways.

A customer, through persistence or by reframing a request, pushes the agent toward something it was not configured to handle. The agent attempts to comply without recognizing the boundary. Separately, the agent encounters an edge case it was not configured for and proceeds rather than escalating. A Head of IT at a healthcare company raised this with me directly during an evaluation: "What protections do you have in place to prevent the agent from going off the rails?" In regulated industries, an agent giving information outside its authorized scope is a liability event. A deployment without a systematic detection layer running on every conversation has no way to identify these failures until a customer complains or a compliance team flags the interaction retrospectively.

3. Goal failures

Every task an AI agent handles has a defined outcome: an order status retrieved, an account verified, an appointment booked, a billing dispute resolved. Whether that outcome was actually achieved is the most direct measure of whether the agent did its job.

Agents fail to deliver the outcome more often than most teams realize. A customer's side question redirects the agent without either party acknowledging the shift. The customer provides incomplete information and the agent proceeds without surfacing the gap. A multi-step process breaks partway through and the agent closes the conversation having completed only part of it. Containment figures record all of these as handled. The customer believes the task is complete. The team has no way to know otherwise until the customer returns.

4. Latency failures

AI agents carry an implicit performance commitment: response time fast enough that the conversation feels natural. When response time stretches from under 2 seconds to 4 or 5 seconds, customers interrupt, lose patience, and request a human agent. A contact center director at a music distribution company described this to me precisely: his agent had stopped performing correctly, and he found out because inbound call volume dried up, not because any monitoring system alerted him. Latency failures typically originate from infrastructure changes: a model update, a backend integration under load, a resource constraint. A latency issue running for hours affects every conversation it touches before it appears in any aggregate report a person would review.

Why transcript-level QA misses all four

QA frameworks built for human agents evaluate conversations by analysing what was said: phrasing, tone, adherence to a script or rubric, accuracy of information provided. This is the right methodology for human agents, where quality is defined primarily by communication.

AI agent quality is defined by what the agent did: which system it contacted, what instruction it sent, whether it followed the correct sequence, whether it achieved the outcome it was assigned. Evaluating a transcript surfaces none of the four failure types above because tool call errors, guardrail breaches, goal failures, and latency spikes do not appear in the words exchanged.

Catching them requires an evaluation layer with access to the full record of what the agent did at each step, running on every conversation, inside the same system as the agent. An external evaluation tool has no access to the agent's decision log, tool calls, or the parameters it sent. It can score what the agent said. It has no visibility into what the agent did.

The final post in this series addresses why that architectural constraint is not solved by adding another tool.

Join me on May 7 for a live walkthrough of a production grade AI agent:

This is Part 2 of a three part blog series by Sumeet Khullar, CTO and Co-Founder at Level AI.
Part 1: The failures hiding inside your AI agent containment numbers (and how to fix them)

Part 3 drops soon!

Frequently Asked Questions

Q1: Why do standard QA reports fail to catch most AI agent evaluation gaps?

A: Standard QA reports are designed for human agents and focus on language quality such as tone, phrasing, and script adherence. AI agent evaluation requires visibility into tool calls, decision sequences, and task outcomes — none of which appear in a transcript.

Q2: What are the most common failure types missed during AI agent evaluation?

A: The four critical failure types are tool call errors, guardrail breaches, goal failures, and latency degradation. Each can silently impact customer experience and compliance without triggering any alert in a traditional QA workflow.

Q3: How does AI agent evaluation differ from traditional contact center QA?

A: Traditional QA measures communication quality; AI agent evaluation measures operational accuracy, including whether the right system was contacted, the correct action was taken, and the intended outcome was achieved. Without this distinction, teams are measuring the wrong things entirely.

Q4: What role does latency play in AI agent performance evaluation?

A: Response times above 2 seconds noticeably degrade the customer experience and increase human escalation requests. Latency failures often go undetected until aggregate metrics shift, making real-time monitoring a critical component of any AI agent evaluation framework.

Q5: How can contact centers detect guardrail failures before they become compliance issues?

A: Detecting guardrail failures requires an evaluation layer that runs on every conversation and has access to the agent's full decision log, not just the conversation transcript. In regulated industries, retrospective review is too slow.