> For the complete documentation index, see [llms.txt](https://learning.contextqa.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://learning.contextqa.com/ai-agent-testing/voice-ai-testing.md).

# Voice AI Testing with ContextQA

Test voice AI agents for conversation accuracy, interruptions, dead air, audio quality, and verified business outcomes in ContextQA.

> **Documentation owner:** [Deep Barot](https://contextqa.com/academy/) · **Last updated:** November 12, 2026

## Quick answer

Voice AI testing evaluates both what an agent does and how a spoken conversation behaves. ContextQA combines conversation criteria with audio judgments and measured voice metrics. Connect a supported voice interface, create realistic caller scenarios, enable the relevant criteria, and inspect audio plus backend evidence before accepting a business workflow.

## Key takeaways

* Test speech behavior separately from text reasoning and transaction correctness.
* Enable the measured audio criteria needed for the project; available criteria may be off by default.
* Use recordings and correlated backend checks to explain failures.

## What should a voice AI test cover?

| Layer                     | Example failure                                       | Guide                                                                                                                      |
| ------------------------- | ----------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| Understanding and context | Loses a correction or a combined name/service request | [Interruptions and context](https://learning.contextqa.com/ai-agent-testing/voice-ai-testing/interruptions-and-context.md) |
| Audio interaction         | Long silence, crosstalk, poor audibility              | [Voice metrics](https://learning.contextqa.com/ai-agent-testing/voice-ai-testing/metrics-reference.md)                     |
| Responsiveness            | Waits too long after a short answer                   | [Dead air and latency](https://learning.contextqa.com/ai-agent-testing/voice-ai-testing/dead-air-and-latency.md)           |
| Business outcome          | Confirms the wrong date or creates the wrong booking  | [Booking and business outcomes](https://learning.contextqa.com/ai-agent-testing/voice-ai-testing/booking-outcomes.md)      |
| Integration and evidence  | Call fails or transcript segments disagree with audio | [Troubleshooting and release review](https://learning.contextqa.com/ai-agent-testing/voice-ai-testing/troubleshooting.md)  |

Start with the [voice quickstart](https://learning.contextqa.com/ai-agent-testing/voice-ai-testing/quickstart.md) for connection and project setup. Use [AI agent testing](https://learning.contextqa.com/ai-agent-testing/agent-evaluation.md) for shared concepts such as personas, rubrics, judges, and backend verification.

## Why a text test is not enough

A typed request can succeed while the spoken version fails. The speech path introduces recognition, timing, audio delivery, interruption, and turn-boundary behavior. A correct transcript also does not establish that the caller heard the reply clearly or that the agent responded at an acceptable time.

Use matched text and voice cases to narrow the investigation. Keep the business request the same, then compare the evidence. A difference identifies a path that needs investigation; it does not alone prove which speech component caused the failure.

## Recommended first suite

Begin with a calm caller, a caller who combines several details, a caller who corrects a date, a short-answer interaction, an interruption during readback, and a failed or unavailable backend action. Define expected outcomes before generation and isolate each case’s fixtures.

For transactional agents, include a criterion for the final user request, a criterion for confirmation, and a verifier for the persisted action. A friendly tone or clear pronunciation must not compensate for an incorrect transaction.

## Video references

[Build an AI Voice Agent in Under 10 Minutes (Amazon Connect + Claude Code)](https://www.youtube.com/watch?v=SMI1IstSKxc\&utm_source=learning.contextqa.com\&utm_medium=docs_referral\&utm_campaign=contextqa_product_education\&utm_content=voice_overview_build) demonstrates constructing an example target agent. It is not a claim that ContextQA’s evaluation workspace builds the target for you.

[My AI Receptionist Booked a Patient 100 Years in the Past](https://www.youtube.com/watch?v=75K0Enqglwc\&utm_source=learning.contextqa.com\&utm_medium=docs_referral\&utm_campaign=contextqa_product_education\&utm_content=voice_overview_context) illustrates why scenario variation and context checks matter. Use the written guides for maintained workflow instructions; video labels and examples can reflect an earlier interface.

## Follow the video chapters with the written guides

The two videos form a build → evaluate → diagnose → retest sequence. The fictional clinic only schedules and cancels appointments; it does not provide medical advice. The build video’s unit and scripted tests establish a starting point, while the evaluation video shows why different caller behavior requires additional voice testing.

| Video chapter                                                                                                                                                                                                                     | What to look for                                                         | Written guide                                                                                                          |
| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------- |
| [01:14 — Describe the agent](https://www.youtube.com/watch?v=SMI1IstSKxc\&t=74s\&utm_source=learning.contextqa.com\&utm_medium=docs_referral\&utm_campaign=contextqa_product_education\&utm_content=voice_chapter_1)              | Define allowed actions and required booking details                      | [Design cases](https://learning.contextqa.com/ai-agent-testing/agent-evaluation/design-test-cases.md)                  |
| [07:51 — Test the built agent](https://www.youtube.com/watch?v=SMI1IstSKxc\&t=471s\&utm_source=learning.contextqa.com\&utm_medium=docs_referral\&utm_campaign=contextqa_product_education\&utm_content=voice_chapter_2)           | Check booking, cancellation, and duplicate attempts against the calendar | [Verify actions](https://learning.contextqa.com/ai-agent-testing/agent-evaluation/verify-tool-actions.md)              |
| [02:57 — Build realistic callers](https://www.youtube.com/watch?v=75K0Enqglwc\&t=177s\&utm_source=learning.contextqa.com\&utm_medium=docs_referral\&utm_campaign=contextqa_product_education\&utm_content=voice_chapter_3)        | Vary how callers supply and correct information                          | [Test caller variation](https://learning.contextqa.com/ai-agent-testing/voice-ai-testing/interruptions-and-context.md) |
| [04:53 — Compare the two callers](https://www.youtube.com/watch?v=75K0Enqglwc\&t=293s\&utm_source=learning.contextqa.com\&utm_medium=docs_referral\&utm_campaign=contextqa_product_education\&utm_content=voice_chapter_4)        | Inspect the failed conversation as well as the score                     | [Review evidence](https://learning.contextqa.com/ai-agent-testing/agent-evaluation/runs-and-regression.md)             |
| [08:08 — Compare typed and spoken input](https://www.youtube.com/watch?v=75K0Enqglwc\&t=488s\&utm_source=learning.contextqa.com\&utm_medium=docs_referral\&utm_campaign=contextqa_product_education\&utm_content=voice_chapter_5) | Separate text success from speech-path behavior                          | [Diagnose voice failures](https://learning.contextqa.com/ai-agent-testing/voice-ai-testing/troubleshooting.md)         |
| [08:54 — Diagnose with MCP](https://www.youtube.com/watch?v=75K0Enqglwc\&t=534s\&utm_source=learning.contextqa.com\&utm_medium=docs_referral\&utm_campaign=contextqa_product_education\&utm_content=voice_chapter_6)              | Read evaluation results before changing the target agent                 | [Use MCP](https://learning.contextqa.com/contextai-mcp/contextai-mcp/mcp-workflow.md)                                  |
| [12:31 — Retest after the fix](https://www.youtube.com/watch?v=75K0Enqglwc\&t=751s\&utm_source=learning.contextqa.com\&utm_medium=docs_referral\&utm_campaign=contextqa_product_education\&utm_content=voice_chapter_7)           | A call can continue while the booking details remain wrong               | [Verify booking correctness](https://learning.contextqa.com/ai-agent-testing/voice-ai-testing/booking-outcomes.md)     |

In the evaluation video, the final spoken confirmation uses May 20, 1930 instead of the requested May 13, 2030. Treat that as evidence of an incorrect confirmation. Establishing what was actually stored still requires a backend check. The same video also shows a shared-slot conflict, so isolate fixtures when reproducing the normal-success scenarios.

Browse the [official ContextQA video channel](https://www.youtube.com/@contextQA_official/videos?utm_source=learning.contextqa.com\&utm_medium=docs_referral\&utm_campaign=contextqa_product_education\&utm_content=voice_channel) for the series. Topics announced as upcoming in a video are not evidence that a later episode or product capability is available.

## Frequently asked questions

### Does a voice quality score prove task completion?

No. Audio quality, conversation correctness, and business outcomes require different evidence. Review all criteria required by your workflow.

### Is there one correct latency threshold for every agent?

No. Define an acceptable limit for the use case and measurement definition. A short conversational response and a disclosed backend operation may have different expectations. Calibrate against representative calls.

### Can I test an agent exposed through a phone number?

Use the **Phone (voice)** connection when appropriate. A telephone path does not establish native integration with every underlying agent vendor. See [connection choices](https://learning.contextqa.com/ai-agent-testing/agent-evaluation/connect-agents.md).

## Extend voice evaluation to business operations

Use [industry playbooks](https://learning.contextqa.com/ai-agent-testing/agent-evaluation/industry-use-cases.md) to define domain-specific scenarios. Evaluate [post-call and multi-agent outcomes](https://learning.contextqa.com/ai-agent-testing/agent-evaluation/end-to-end-outcomes.md) separately from the conversation, and follow [pre/post-deployment testing](https://learning.contextqa.com/ai-agent-testing/agent-evaluation/deployment-lifecycle.md) and [production analytics](https://learning.contextqa.com/ai-agent-testing/agent-evaluation/production-analytics.md) for lifecycle and measurement boundaries.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://learning.contextqa.com/ai-agent-testing/voice-ai-testing.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
