How voice agents handle interruptions and barge-in

Voice agent interruption handling is three decisions made throughout a call: whether someone is speaking, whether the caller has finished, and whether speech over the agent should make it stop. The agent must stop quickly on a real interruption, then carry on from what the caller actually heard.

On the phone, people talk over each other all the time. They say "mm-hm" to show they are listening, cough, correct a date halfway through a sentence, or answer before the question has finished. A voice agent has to decide which of those sounds means "stop talking".

Swipe sideways to see the whole diagram.

Key facts

• Silero VAD handles an audio chunk of 30 ms or more in under 1 ms on one CPU thread (Silero README).
• Pipecat’s Smart Turn model has about 8 million parameters and covers 23 languages (Smart Turn README).
• Smart Turn takes 16 kHz mono audio, while G.711 phone codecs are listed at 8,000 Hz (Smart Turn README, RFC 3551).
• LiveKit marks its text turn detector as deprecated in favour of an audio turn detector (LiveKit).
• LiveKit’s interruption options include a minimum duration and a minimum word count (LiveKit).
• Cekura’s public turn-taking set has 45 calls from 15 scripts (Cekura).

What are the three decisions?

Interruption handling is three separate decisions that run throughout the call. Voice activity detection (VAD) decides whether someone is speaking. End-of-turn detection decides whether the caller has finished. Interruption logic decides whether speech that arrives while the agent is talking should make it stop. Each can be tuned on its own.

How does voice activity detection work?

A VAD model, such as Silero VAD, listens to short stretches of audio and says whether speech is present. Its README says it handles a chunk of 30 ms or more in under 1 ms on one CPU thread. It is the first gate for everything else. What it cannot do is tell a cough from a word, or "mm-hm" from "no". It only says whether a voice is present.

The settings that matter are how sure the detector must be and how long speech must last before it counts. Pipecat's detector, for example, has a confidence setting and separate start and stop times. Too sensitive, and the agent stops for a door closing or a passing bus. Too dull, and callers have to raise their voices. Phone audio adds a check of its own. RFC 3551 lists the G.711 phone codecs at 8,000 Hz, and Silero VAD says it supports 8000 Hz and 16000 Hz, so confirm your detector copes with 8 kHz audio.

How does a voice agent know the caller has finished?

Silence alone is a poor sign that someone has finished. People pause mid-sentence, especially when reading out a number or thinking of a date. A fixed silence timer forces a trade. Keep it short and the agent cuts people off, but make it long and every reply feels slow. Two kinds of model now sit on top of VAD to make a better call.

Audio models listen to how something was said. Pipecat's Smart Turn is an open model that uses Whisper Tiny as a base with a linear classifier layer. Its README gives about 8 million parameters, 23 languages and a BSD 2-clause licence, and reports as little as 10 ms per decision on some CPUs. It runs once VAD hears silence, over the whole of the caller's turn. Its authors note that VAD "can't take into account the actual linguistic or acoustic content of the speech".

Because Smart Turn works from the audio, its README says it can take "subtle prosody cues" into account, so a trailing "I can't seem to, um..." is a good case to try. It takes 16 kHz mono audio, so check it on real phone audio before you trust it there.

Text models read the transcript instead. LiveKit's text turn detector is an open-weights model based on Qwen2.5-0.5B-Instruct. It reads what speech-to-text produced and judges whether the sentence sounds complete. It sees words, not tone, and it is only as good as the transcript. LiveKit's turn detector documentation now marks the text model as deprecated in favour of an audio turn detector, which "encodes user audio directly, capturing both what is said and how it's said".

Backchannels and false stops

A backchannel is a short sound that means "go on", such as "mm-hm", "yeah" or "right". An agent that treats each one as an interruption stutters through every long answer. The usual defences are a minimum length of speech, a minimum number of words before speech counts as an interruption, and a short wait to see whether the caller keeps going. LiveKit's interruption options include both a minimum duration and a minimum word count, the second used only when speech-to-text is running.

Some false stops will still get through. A cough, a laugh or a dog barking can pass every filter. What matters then is recovery. LiveKit raises a false interruption event when no real speech follows, and by default "the agent resumes speaking from where it left off after a false interruption". Without that, a cough leaves the agent silent while the caller waits.

Why does a voice agent interrupt itself?

The agent can also interrupt itself. If the caller is on a speakerphone or a car handsfree kit, the agent's own voice may come back down the caller's side of the call. If echo cancellation misses it, VAD hears speech and the agent stops mid-word for no reason a listener could hear. The network side has its own standard, ITU-T G.168 on digital network echo cancellers, but what happens in a car or a room is a separate path that you have to test. Test with echo added to the caller's audio, not only with clean recordings.

After the stop

Stopping is half the job. The agent's history may still hold the whole reply it planned to say. LiveKit's documentation says that when interrupted, the agent "automatically truncates its conversation history to include only the portion of the speech" that the caller heard. Without that, the agent later refers to things the caller never heard, such as "as I mentioned, the excess applies to this claim".

Then check for lost words. If what the caller says over the agent is clipped at the start or dropped, "No, Thursday, not Tuesday" could arrive as "Thursday, not Tuesday", or not at all, and the booking would go ahead on the wrong day.

What to test

• Real interruptions early, midway and in the last second of a long reply. Frustrated people cut in more often, so include angry callers who talk over the agent and repeat themselves.
• Backchannels such as "mm-hm", "yeah" and "okay" during a long answer. The agent should keep going.
• Sounds that are not speech, such as a cough, a laugh, a door or traffic. The agent should keep going, or resume if it stopped.
• Pauses mid-turn, such as a policy number read in two halves. The agent should wait.
• The agent's own voice echoed back on the caller's side.
• Corrections during a read-back, such as a caller fixing a date or a number.

Run all of it over phone audio such as G.711 at 8,000 Hz, with the background noise your callers really have. RingTru’s simulated callers ring your agent with interruptions, accents and noise. For why typed tests miss this, see text vs audio testing.

What to measure

• Time to stop. From the start of the caller's speech to the agent's last sound. Report the median and the 95th percentile, so the slowest stops show up and do not vanish into an average.
• False stops. How often the agent stops for a backchannel, a cough or noise, per hundred such events, and how often it then resumes properly.
• Missed stops. Real interruptions where the agent kept talking.
• Lost words. Words the caller said during the overlap that never reached the transcript.
• Unheard references. Later turns that mention words the caller never heard.

Public benchmarks help with a shortlist. Cekura's turn-taking benchmark on Hugging Face tests four behaviours: waiting through pauses, replying when the caller is done, stopping when the caller cuts in, and carrying on through "mm-hmm". Its public set has 45 calls from 15 scripts, in quiet-room conditions with synthetic voices, and it scores cut-ins live by measuring whether and how quickly a model stops.

It is free to use under CC BY 4.0 and a fair baseline. It is not your phone line, your callers or your prompt, so run the same cases against your own agent before you rely on a number.

Model details and settings change often. Everything here was checked against the linked pages on 27 October 2026. Related: what makes or breaks a voice agent, speech-to-speech or a chained pipeline, and the glossary entries for barge-in and backchannel.