ElevenLabs’ Eleven Realtime runs a whole voice agent from one model

2 hours ago

@elevenlabsSubscribe

ElevenLabs launched Eleven Realtime: Speech to Speech models where one model is the whole voice agent, in ElevenAgents and through an OpenAI-compatible Realtime API. v1 Mini is the low-cost default; v1 Max reasons before it answers, for support and multi-step tool use. Rolling out gradually. https://elevenlabs.io/blog/realtime https://elevenlabs.io/realtime https://elevenlabs.io/docs/eleven-agents/customization/realtime-models https://elevenlabs.io/docs/eleven-agents/guides/openai-realtime-compatibility

Ask

Ask about this presentation

Answers are generated from this presentation.

Chapters

  1. 0:00Eleven Realtime
  2. 0:19Mini and Max
  3. 0:37One dropdown in ElevenAgents
  4. 0:52OpenAI-compatible Realtime API
  5. 1:08Accuracy and speed
  6. 1:21Where Realtime doesn't fit yet
  7. 1:39Your setup
Show transcript

Eleven Realtime

ElevenLabs

Eleven Realtime runs your whole voice agent from one model

New Speech to Speech models in ElevenAgents and the Realtime API

Speech recognition
Turn-taking
LLM
Text to speech
Eleven Realtime
elevenlabs.io/blog/realtime

ElevenLabs launched Eleven Realtime, Speech to Speech models where one model is the whole voice agent. Speech recognition, turn-taking, the LLM and the voice come tuned together, so you stop assembling and testing that stack yourself.

Mini and Max

ElevenLabs

Mini for conversation, Max for multi-step work

v1 Mini The default · lowest latency and cost · assistants, tutors, characters
v1 Max Reasons before it responds · customer support, lead qualification · costs more
Tool-use tasks completed
Max
87.1%
Mini
60.3%
elevenlabs.io/blog/realtime · elevenlabs.io/docs Realtime models

Eleven Realtime v1 Mini is the cheaper default, for assistants, tutors and characters. v1 Max reasons before it answers. Max completes almost nine in ten tool-use tasks, against six in ten for Mini, so Max suits support calls and multi-step workflows.

One dropdown in ElevenAgents

ElevenLabs

In ElevenAgents, set Voice mode to Realtime

Prompt, voice, tools and knowledge base carry over. Rolling out gradually.

ElevenAgents agent settings with the Voice mode switch, Realtime or Custom
elevenlabs.io/docs/eleven-agents/customization/realtime-models

In ElevenAgents, you set an agent’s Voice mode to Realtime and pick Mini or Max. Your prompt, voice, tools and knowledge base carry over. Realtime is rolling out gradually, so some workspaces don’t have the switch yet.

OpenAI-compatible Realtime API

ElevenLabs

OpenAI Realtime code now points at ElevenLabs

Same events, OpenAI’s SDK as is · WebSocket only

// @openai/agents-realtime
new OpenAIRealtimeWebSocket({
  url: "wss://api.elevenlabs.io/v1/convai/realtime",
});
new RealtimeSession(agent, {
  transport,
  model: "eleven_realtime_v1_mini",
});
elevenlabs.io/blog/realtime · elevenlabs.io/docs OpenAI Realtime API compatibility

If your app already runs on OpenAI’s Realtime API, Eleven Realtime speaks the same events, so OpenAI’s SDK works as is. You change the endpoint and the model name. It connects over WebSocket only, so a WebRTC app has to move first.

Accuracy and speed

ElevenLabs

Max gets almost every spoken question right, fast

Big Bench Audio · 98.8% correct · first audio in 0.57 seconds

Accuracy against time to first audio for speech to speech models, with Eleven Realtime v1 Max top left
elevenlabs.io/blog/realtime · Artificial Analysis leaderboard, October 2026

Asked reasoning questions out loud, Max gets nearly all of them right and starts talking in under six tenths of a second, ahead of every model that scores near it. Mini gets about three in four right.

Where Realtime doesn't fit yet

ElevenLabs

Some agents stay in Custom mode

Your own model stackCustom LLM, Amazon Bedrock, a specific TTS model, or text only
Compliance and regionZero Retention Mode and HIPAA: Max only · India and Singapore data residency: Mini only
elevenlabs.io/docs/eleven-agents/customization/realtime-models

Agents on a custom LLM or Amazon Bedrock, or tied to a specific TTS model, stay in Custom mode, and so do text-only agents. Under HIPAA or Zero Retention Mode only Max works, and in India or Singapore data residency only Mini does.

Your setup

ElevenLabs

Moving an OpenAI Realtime app: OpenAI Realtime API compatibility guide, in the ElevenLabs docs

Is your voice agent built in ElevenAgents, or on OpenAI’s Realtime API?

elevenlabs.io/docs/eleven-agents/guides/openai-realtime-compatibility

The steps for moving an OpenAI Realtime app are in the OpenAI Realtime API compatibility guide. So tell me, is your voice agent built in ElevenAgents today, or on OpenAI’s Realtime API?