How to test a Retell AI voice agent

Retell gives you four ways to test, and they work best in order. Debug in the text playground, save the scenarios that matter as simulation tests, then talk to the agent in a browser web call and finish on a real phone. After each run, read the call history and post-call analysis. If you want to keep score over time, pull the same data by API or webhook. Everything here comes from Retell's public docs, linked beside each feature.

Key facts

• Retell lists four test methods in its testing overview, and says simulation testing is the only one that grades a run.
• Retell simulations time out after 10 minutes and are cut off past 400 utterances, per the simulation testing page.
• Retell simulations always fake transfers and SMS, so only a phone call tests them, per the simulation testing page.
• Web calls cannot transfer to a phone number or send SMS, per the web call testing page.
• Retell agents will not hang up by default, so you set the end call conditions, per the end call page.

What does Retell give you to test with?

The testing overview lists four methods and suggests an order. Each step is slower than the last and closer to a real call, so it pays to start at the cheap end.

• The LLM Playground is a text chat in the agent editor. You type each turn or let an AI play the caller, and tool calls and state changes show inline. It grades nothing. See Manually test your agent.
• Simulation testing runs saved scenarios with success criteria, singly or as a batch. The overview says it is the only method that grades a run. See Automatically test your agent.
• Web call testing has you speak to the agent in the browser. You hear real audio, latency and interruptions without needing a phone number. See Web call testing.
• Phone call testing is a real call, so it covers carrier audio, keypad tones and transfers. See Phone call testing.

How do Retell simulation tests work, and where do they stop?

A simulation is a text conversation between your agent and an AI caller. You describe who the caller is and what they want. You also add success criteria and set any dynamic variable values. Retell marks the run pass or fail and gives a written reason, per the simulation testing page.

The limits are worth knowing before you lean on it. It is text only, so it says nothing about speech recognition, accents or timing. It supports single-prompt, multi-prompt and Conversation Flow agents but not custom LLM agents, and the overview adds that speech-to-speech agents cannot be simulated through text. A run times out after 10 minutes and is cut off past 400 utterances, as the same page explains. Those cut-off runs end as errors, not as fails.

The grading has quirks of its own. Retell judges all your criteria together in one pass, and a single unmet criterion fails the run. If you want an answer per criterion, write each check as its own test case. The caller and the grader are both language models, so the docs suggest judging a scenario on its pass rate across several runs and ignoring one odd failure.

Treat a pass rate from a handful of runs with care. Seven passes in ten runs is 70 per cent, but the true rate could plausibly sit anywhere from roughly 40 to 90 per cent. Report such a rate with a confidence interval, and use the Wilson method, which NIST documents for proportions and which behaves better than the simple textbook formula when the sample is small. State the number of runs beside the percentage, and say if repeats used the same script.

Tools need care. Retell says unmocked custom functions and code tools call your live endpoints, and calendar tools do too, so a test can create a real booking. MCP tools ignore mocks altogether. Mock anything that touches production before you start a batch. Transfers and SMS are always faked in simulation, which means only a phone call will exercise them.

For automation, the same page describes an API route. You create test cases, start a batch, poll until it completes, then read the pass, fail and error counts and each run's transcript. Even a single case runs as a batch of one.

What should you check on a real call?

Use the web call to judge how the agent sounds. The page lists latency, turn-taking, whether the agent stops when you talk over it, pacing and pronunciation. You can also play a recorded voicemail greeting into the microphone to see if detection fires. Web calls cannot transfer to a phone number or send SMS.

For callers with accents, noise and interruptions at scale, RingTru's simulated callers ring an agent the way a customer would. See also text vs audio testing. Anything else needs the phone test. It validates keypad handling, warm and cold transfers, voicemail and IVR detection, SMS and real carrier audio. You can attach a draft version to a number without publishing it. Newer workspaces need identity verification before outbound test calls.

Which agent settings are worth checking?

These are documented on the Create Agent reference unless noted. Read the value if you like, but the test is a call.

• Responsiveness runs from 0 to 1 and controls how quickly the agent replies. Check that it leaves room for a caller who pauses to find a number.
• Interruption sensitivity also runs from 0 to 1, and at 0 the agent is never interrupted. Retell's background speech page says higher values make the agent easier to interrupt. Try a real interruption, a "mm-hm" and a cough. Our guide on interruption handling covers the cases.
• Backchannel is a boolean that stays off unless you set it, with a frequency from 0 to 1 and an optional word list. If you switch it on, listen to whether the "yeah" lands at natural moments.
• Denoising has three modes on the background noise page. The default removes noise but leaves loud background speech, and the strongest mode can distort the audio. Test with a television or a second voice in the room.
• Silence and reminders: the reference describes a reminder trigger, a maximum reminder count and an option to end the call once the caller stays silent. Find out what a caller who goes quiet actually hears.
• Voicemail and IVR: the voicemail page covers hanging up or leaving a message, detected in the first 3 minutes of a call. Try both a voicemail greeting and an automated menu.
• Transfers: the transfer page lists cold, warm and agentic warm transfers and says they work on phone calls only. Check what happens when nobody answers and what the destination hears.
• End call: per the end call page, the agent will not hang up by default, so you set the conditions. Say goodbye and see if it ends cleanly. Then say "thanks" halfway through a task and see whether it hangs up when it should not.

Where do you read Retell test results?

The dashboard's call and chat history gives you the transcript with tool calls and state changes, a recording that jumps to the line you click, end-to-end latency, a conversation analysis panel and detail logs. Filters include outcome and sentiment, plus disconnection reason and latency.

The Get Call API returns the same material for scripts. That covers the transcript and a latency object with sections such as end-to-end, speech recognition, language model and speech synthesis. It also returns the recording URL, the disconnection reason, a transcript that interleaves tool calls and their results, and a public log URL with per-turn latency tracking.

Post-call analysis has four built-in fields: summary, user sentiment, call successful and in voicemail. You can add custom fields typed as boolean, text, number or selector, as the overview page shows. A language model infers sentiment from the transcript and ignores tone of voice, so use it as a rough flag.

For live results, webhooks cover call started, ended, analysed, transcript updates and four transfer events. Subscribe to the analysed event if you want the analysis, because the consumption page says the ended event leaves it out. Check the x-retell-signature header and make handlers safe to repeat, since Retell retries and may send duplicates.

A short list of calls to run

1. A plain, successful call from start to hang-up, to set a baseline.
2. A caller who interrupts early, midway and at the end of a long reply.
3. A caller who says "mm-hm" and coughs during an answer.
4. A caller who reads a long number with a pause in the middle.
5. A caller who goes silent, then returns.
6. A transfer to a person who answers, and one to a number that does not.
7. An outbound call that reaches voicemail, and one that reaches an automated menu.
8. A call with background television or a second speaker.
9. A caller who changes their mind, or corrects a detail already read back.
10. A caller who ends the call abruptly, to check the disconnection reason and the analysis.

Run the first three as simulations for the logic, then repeat them on a web call for the audio. Anything on the list that involves a transfer or a menu needs the phone. See the guides by platform for other vendors and voice agent testing for the wider method. To compare Retell with its nearest rivals, see Vapi vs Retell vs Bland, or read how to test a Vapi voice agent.

Retell's docs change often, so everything here was checked against the linked pages on 27 October 2026.