UK accent speech recognition benchmark on phone audio

Public leaderboards mostly test clean recordings. A phone agent hears accents over phone-quality audio (8 kHz, the narrow band a phone line carries). This benchmark will measure that.

1Accent recording
28 kHz phone codec
3Line noise
4Speech model
5Word error rate

Round one

Round one will report word error rate by accent group in three conditions: clean, phone line, and phone line with noise.

Round one has not run yet. Results will go here.

Models: Deepgram Nova-3, AssemblyAI Universal-3.5 Pro, OpenAI gpt-4o-transcribe, ElevenLabs Scribe v2, Speechmatics Enhanced, Whisper Large V3 (open), NVIDIA Parakeet TDT 0.6B v3 (open). Accent groups: Scottish, Welsh, Northern, Midlands, Southern, Irish.

Three rounds

Planned

Transcription

Word error rate on the same clips in all three conditions.

Later

Streaming

Time from the end of speech to the first final word, as the median and the slowest 1 in 20.

Later

Whole calls

Speech-to-speech models on scripted calls, judged on our report rubric, every transcript published.

The method

We wrote the method before the first run and dated it, so any change is visible.

Read the method

Audio: Read speech from the UK and Ireland English dialect set (OpenSLR 83), about 17,900 recordings from Irish, Midlands, Northern, Scottish, Southern and Welsh speakers, licensed CC BY-SA 4.0. Unscripted speech from the Edinburgh International Accents of English Corpus.
Phone line: Each clip is resampled to 8 kHz and passed through a G.711 A-law encode and decode, the codec most UK landline calls use. The noise condition mixes in street and kitchen sound at 10 dB signal-to-noise. All phone audio is labelled as simulated.
Scoring: Word error rate after the same text normaliser is applied to the reference and to each output. We use the English normaliser from Whisper, as the Open ASR Leaderboard does, so our clean figures can be compared with theirs. Results per accent group, with 95 percent bootstrap confidence intervals. Groups with too few clips are shown, flagged and left out of rankings.
Versions: Every result names the model ID or dated version, the API region, the run date and the commit of our runner. A model that changes under the same name gets a new row.
Fairness: Each vendor sees its own results ten working days before publication and may reply. Replies are published beside the results. We do not sell a speech model and take no payment from any vendor we test.
Reproduce it: The clip lists, phone-line script, normaliser settings and scoring code will be public when round one ships. Open data

Leaderboards we cite

They measure different things. We link to them.

None of these publishes results for UK accent groups over phone-quality audio. Checked 26 October 2026.

Run a test on your own agent.

Get a free report Book a call