Pipecat v1.16.0 now hears how your caller sounds, not just their words

Pipecat v1.16.0 hears your caller, not just their words. The new Pipecat Voice Reader listens each time the user stops talking and reports their tone, laughter, speaking rate and sarcasm, on the CPU.

Hearing the caller.

The new Perception Reader asks several classifiers at once, and the LLM gets every answer. Pipecat Tone reads the words, Voice Reader hears the voice, and Hume or Google answer questions you write.

Pipecat used to keep only the last reading of a user's turn. Now it merges them all, so sarcasm in the first sentence still counts. And the match style preset laughs at jokes more reliably.

Speaker diarization now lands in one field on every transcription. ElevenLabs, AWS Transcribe and OpenAI's diarize models join, and RTVI passes the speaker to your client. AssemblyAI, Speechmatics and Nvidia users should check their code.

New services.

Gemini's Interactions API is now a Pipecat LLM service, and Google stores no conversation state. A call can switch to it from the older Gemini service mid-call, and a Vertex AI version runs Gemini 3 and later.

Zoom's AI Voice Agent joins as a speech-to-speech service. It detects turns and barge-in itself, so your pipeline needs no VAD, and it can run the conversation on your own OpenAI or Anthropic account.

HeyGen Voice arrives as WebSocket and HTTP text-to-speech services. Its per-character timing means an interruption keeps only the words the caller actually heard, and the HTTP service takes professional voice clones.

Rime now streams speech-to-text with interim results, and Sarvam adds a REST service. Soniox gets echo cancellation: it follows the bot's reply word by word, so it never transcribes the bot as the user.

Audio and telephony.

Pipecat dropped resampy, and with it numba and llvmlite, which roughly halves the default install. soxr is out of the core too. A new Pipecat resampler package keeps output chunks in step with the input.

Several fixes target phone calls at 8 kilohertz. Whisper, Moonshine and FunASR stop mistranscribing them, Gemini text-to-speech plays at the right speed, and bot audio stops arriving in choppy bursts.

A new jambonz serializer connects Pipecat through jambonz's listen verb. It reads the sample rate from the call and turns keypad digits into frames. Turn on streaming in the verb, or callers hear nothing.

Evals and deadlines.

Pipecat evals can now grade calls they didn't drive. A rubric keeps only the judge, the success test and the judged metrics, so recorded production calls meet the same bar as your simulations.

Hume shuts down its text-to-speech API on November thirteenth, so Pipecat deprecates its Hume service. A Gemini preview voice retires on the seventeenth, moving Pipecat's default to Gemini 3.8 Flash Lite, and Fish Audio retires s1 at year's end.

I'd add Voice Reader beside Pipecat Tone first, on a support line where callers get frustrated, and keep it on a branch while it's alpha. The upgrade steps are in the Pipecat v1.16.0 release notes.