ElevenLabs Scribe v3 Live Tries to Stop Voice Agents Cutting Callers Off

The most interesting part of ElevenLabs' new speech recognition release isn't the accuracy headline. It's the claim that a voice agent can tell when a caller has actually finished talking, rather than guessing from a silence timer. If that holds up in production, it addresses one of the most common complaints about phone bots.

ElevenLabs has released two speech-to-text models, Scribe v3 and Scribe v3 Live. Both are aimed at difficult audio, such as people talking over each other in meetings or phone calls with background noise. Scribe v3 handles pre-recorded files, while Scribe v3 Live targets low-latency, real-time use. Both are available through the ElevenAPI and the ElevenAgents platform.

Where Scribe v3 sits on accuracy, speed and price

ElevenLabs says Scribe v3 reduces word errors by about 25 percent compared with its predecessor across 102 languages. The company also reports a significant German accuracy gain and more reliable speaker diarization, the task of working out who said what, in larger group conversations. Scribe v3 Live is the first model in this family to offer that diarization in real time.

Those are vendor claims, but independent numbers exist. In tests run by Artificial Analysis, Scribe v3 ranks among the most accurate transcription models available. It beats offerings from Microsoft, Mistral, xAI, Google and OpenAI, and only a preview model from Alibaba scores higher.

The trade-off is throughput and cost. Microsoft and xAI models run many times faster, with marginally higher error rates, and cost roughly half as much. Scribe v3 lands mid-pack on both speed and price. If your job is transcribing thousands of hours of archived calls on a budget, the accuracy edge probably doesn't justify the bill. If every word in a regulated or high-stakes recording matters, the calculation looks different.

Keep in mind what the benchmark doesn't measure. It scores word errors on recorded files, so it says nothing about the diarization gains, the rare-language improvements or the behavior of Scribe v3 Live. Those are the claims that matter most for voice agents, and they need their own testing.

Why turn-taking is the real problem for voice agents

Most voice agents still decide that a caller has finished by waiting for a fixed stretch of silence. That works for short, complete sentences. It breaks when someone pauses to think, trails off mid-thought, or reads out a long string of digits such as an account or policy number. The agent either cuts the caller off or waits awkwardly after they've finished.

Scribe v3 Live is designed to fix this by analyzing conversational context alongside the audio signal. Instead of asking only "has there been silence for X milliseconds?", the model can weigh whether the utterance so far looks like a complete thought. My read is that this is the right direction. Pause timers are a blunt tool, and making the end-of-turn decision depend on language content should reduce both premature interruptions and dead air. Still, ElevenLabs hasn't published figures on how often this happens or how the approach performs with accents, noisy lines or hesitant speakers, so developers should measure it on their own call recordings.

Controls for domain vocabulary and output format

Beyond turn detection, the models give developers more say over the final transcript. You can preload thousands of specialized terms, such as brand names or medication names, so the model doesn't mishear domain vocabulary. Formatting rules let you specify how numbers and dates appear in the output. A multilingual mode handles speakers who switch languages mid-sentence.

For enterprise buyers, the deployment options matter as much as the model. ElevenLabs offers isolated servers in the EU and a zero-retention mode that deletes data immediately after each request. Teams in healthcare, finance or customer support often need to settle questions about where audio is processed and how long it persists, so those options will carry weight in procurement conversations.

Enterprise expansion and what to watch

ElevenLabs is pairing the release with a broader enterprise push. Under an expanded partnership with Nvidia, the company is integrating the open Nemotron 3.5 Super VL language model into ElevenAgents, securing additional compute capacity running on Nvidia hardware, developing local voice control for robots, and running a phone agent for Poland's public health system. It has also announced an alliance with consulting firm McKinsey, a partner program for system integrators, and a marketplace where companies including Salesforce, IBM and Zendesk offer ready-made voice agents.

Here's the thing: a model can top an accuracy chart and still be the wrong choice for a high-volume batch job. The better question is which workload you're running. For batch transcription, the Microsoft and xAI comparisons suggest price and speed will dominate. For live voice agents, the open questions are whether context-aware turn-taking holds up outside the demo, and how latency behaves under real call volumes. Those are the numbers worth requesting before committing to a rollout.