How to test a Pipecat voice agent

To test a Pipecat agent, start with Pipecat's own evals, which run scripted and simulated conversations against your real pipeline. Then switch on metrics and tracing so every call shows where time went. Finish with real phone calls over your telephony transport. Pipecat gives you more built-in help here than most frameworks, and the pipeline's structure makes faults easy to locate.

Key facts

• Pipecat Evals run scripted and simulated scenarios against your real pipeline, per the evals overview.
• Repeated eval sweeps always exit with success, so you must set your own pass threshold, per the eval suites page.
• Voice activity detection start_secs and stop_secs both default to 0.2 seconds, per the speech input page.
• Twilio Media Streams carries 8kHz mono audio, per the Twilio guide.
• Pipecat Flows does not support speech-to-speech models, per the Flows introduction.

Why does Pipecat's structure help testing?

A Pipecat agent is a pipeline of frame processors. Each processor does one job, such as speech-to-text or text-to-speech, and passes data along as frames. The docs say each frame gets an automatic name like TranscriptionFrame#1, which helps with debugging.

So when a call goes wrong, you can ask which stage produced the bad frame. A wrong date in the transcript points at speech-to-text. A right transcript with a wrong reply points at the language model. The same page says system frames are ordered with other system frames and that interruptions do not discard them, so keep that in mind when you test barge-in.

Can I run automated tests?

Yes. Pipecat Evals describe a conversation and the behaviour you expect, then run it against your real agent. The page says it is "the same pipeline, the same services, the same code." There are two kinds of scenario.

Scripted scenarios fix the caller's turns. Each turn lists events the agent must produce, and the docs say they must arrive in the order listed. You can check for exact text with text_contains or text_excludes, judge meaning with a plain-language eval: criterion, check that a tool was called with the right arguments, set a time limit with within_ms, and send keypresses with dtmf:. A send_after: setting lets you anchor a turn to an event to test barge-in.

Simulated scenarios use an LLM as the caller. You give it a persona: and a goal:. It plays the conversation alone and hangs up once it has what it came for. A judge reads the transcript and the agent's tool calls, and the docs state that "the judge decides the outcome." The judge sees each call's name and arguments but not its result, so check results another way.

Two modes matter. Text mode skips speech recognition and speech synthesis, so it tests logic cheaply. Audio mode synthesises the caller's speech and transcribes the agent's audio, so it tests the whole chain. The overview page lists limits: speech-to-speech agents support audio mode only, and WebSocket-streaming speech services are not supported in audio mode.

To get a pass rate rather than one lucky run, use eval suites. A repeat: setting runs every agent and scenario pair several times and reports a rate for each. The same page notes a repeated sweep always exits with success, so you must read the results file and set your own threshold.

What do the metrics tell me?

Pipecat's metrics page lists performance metrics you turn on with enable_metrics=True in PipelineParams. These are time to first byte, time to first audio for speech synthesis, time to first answer token for language models, processing time, and text aggregation time. Usage metrics, switched on with enable_usage_metrics=True, report language model tokens and speech synthesis characters.

Use them to find the slow stage. A slow first reply is rarely the whole pipeline's fault, and it is usually one service. Run every test call with metrics on and keep the output with the recording. For why gaps matter on a call, see turn-taking and latency.

How do I trace a whole call?

Pipecat supports OpenTelemetry tracing. You install the tracing extra, call setup_tracing() with an exporter, and pass enable_tracing=True to the worker. The docs describe a conversation span that holds turn spans, and each turn holds speech-to-text, language model and speech synthesis spans.

For reading these traces, see voice agent call traces with OpenTelemetry. The attributes include the transcript, the model's input and output, tool names, token counts, and whether a turn was_interrupted. That gives you one record per call to compare against what the caller said. The page lists exporters for tools such as Jaeger and Langfuse. Metrics also need to be enabled for some of the service figures.

How do I test a Flows agent?

Pipecat Flows structures a conversation as a graph of nodes. Each node has its own prompt and its own tools, and functions move the call to the next node. The page says Flows needs a text language model with function calling in a chained pipeline, and does not support speech-to-speech models.

The Flows introduction page does not cover testing, so this part is our reading. Because the language model chooses which function to call, the graph is only as reliable as those choices. Write scenarios that push the caller towards each branch, including the awkward ones: a caller who changes their mind, gives details out of order, or asks for something the current node cannot do. Check that the right function fires with the right arguments.

What should I tune before testing turn-taking?

Pipecat's speech input page gives voice activity detection settings start_secs and stop_secs, both defaulting to 0.2 seconds. It says changing stop_secs means re-benchmarking the speech-to-text latency figures. For end-of-turn decisions, Smart Turn is the default stop strategy, and it has its own stop_secs, which acts as a fallback when the model marks a turn incomplete.

Interruptions are on by default. A minimum-words setting, which you must configure, can let short sounds like "okay" or "yeah" pass without stopping the agent. After a stop, the context keeps only the words actually spoken. Test all of this with callers who pause mid-number and callers who say "mm-hm". More in interruption handling.

How do I test on a real phone line?

Pipecat documents WebSocket, WebRTC and SIP routes, with Twilio, Telnyx, Plivo and Exotel for WebSocket. The Twilio guide shows a TwiML stream that opens a WebSocket to your bot, a serializer that ends the call when the pipeline ends, and local testing through ngrok. It covers inbound calls to a number and outbound calls placed through Twilio's API.

Phone audio is narrow. The Twilio page says Media Streams carries 8kHz mono audio and advises setting the input and output sample rates to 8000 Hz to avoid resampling. A bot that works on a laptop microphone can behave differently here. The production page adds that carriers can drop calls that stay silent too long, so test a long silence. See also phone audio, accents and noise.

Calls to run

• A caller who gives details in the wrong order, then corrects one.
• A caller who interrupts early, midway and near the end of a long reply.
• A caller who says only "mm-hm" and "okay" while the agent talks.
• A caller who pauses halfway through reading out a number.
• A request the agent should refuse or hand over, and a check that it does.
• A long silence, to see how the agent and the carrier cope.
• Each branch of a Flows graph, with the function arguments checked.
• The same scenario repeated many times, because one pass proves little.
• A call over the real phone route with traces and metrics on, not only the eval transport.

Keep the transcript, the trace and the recording for every run, so a failure can be traced to a stage. RingTru's simulated callers can ring the real phone route with accents, noise and interruptions. For why text evals are not enough, see text vs audio testing. A sibling guide covers how to test a LiveKit voice agent. Pipecat's docs change quickly, and everything here was checked against the linked pages on 27 October 2026. For the wider picture, see voice agent testing and the other guides by platform.