Pipecat v1.15.0 teaches your voice bot to laugh and match the caller’s mood
2 hours ago
Pipecat v1.15.0 adds expressive conversation (alpha): the LLM writes tags for tone, laughs, pauses and stressed words, and nine TTS providers perform them. The LLM also hears how the user sounded, Pipecat Tone reads emotion and sarcasm, and Tavus avatars change expression. Also: a classifier that decides when the user is done, stop strategies that now must all agree, AssemblyAI TTS, faster OpenAI Responses after interruptions and Bedrock reasoning. Release notes: https://github.com/pipecat-ai/pipecat/releases/tag/v1.15.0 CHANGELOG: https://github.com/pipecat-ai/pipecat/blob/v1.15.0/CHANGELOG.md Full diff: https://github.com/pipecat-ai/pipecat/compare/v1.14.0...v1.15.0 Examples: https://github.com/pipecat-ai/pipecat/tree/v1.15.0/examples/expressive-conversation Docs: https://docs.pipecat.ai
Ask about this presentation
Chapters
0:00Pipecat v1.15.0
0:11Expressive conversation
0:12The tags
0:24TTS providers
0:35The LLM learns the tags
0:48Expression policy
0:59Perception
1:08Pipecat Tone
1:19Tavus
1:28Turn-taking
1:29Classifier turns
1:40Stop strategies agree
1:52Turns end on the latest words
2:03Services
2:04AssemblyAI TTS
2:16Responses after interruptions
2:29Bedrock reasoning
2:43Upgrade notes
Show transcriptHide transcript
Pipecat v1.15.0 adds expressive conversation. Your LLM writes tags into its reply, and the TTS performs them as a warm tone, a laugh or a pause.
Expressive conversation.
A delivery tag such as excited holds until the next one. Others add a laugh, a pause or a stressed word. It's alpha, so expect the syntax to change.
ElevenLabs, Cartesia, Fish Audio and MiniMax perform them, and so do five more. Only on certain models, and a tag a service can't perform is dropped with a warning.
Your prompt doesn't have to describe the tags. Pipecat writes them into the system instruction, in each provider's own words, and rewrites it when you switch TTS.
Guidance presets tell the bot when a tag fits. One matches the caller's mood and energy, one slows down for codes, and one adds an occasional breath.
It works in the other direction too. Each user turn can carry a tone label with a confidence, written at the head of the message.
Pipecat Tone does both jobs on the CPU. Its reader judges emotion and sarcasm in each turn, and its tagger adds tags to plain LLM replies.
Tavus video avatars pick up the same tags. When the bot speaks warmly, the replica's face shows happiness, timed to the words.
Turn-taking.
If your LLM ignores the turn completion markers, a classifier can judge instead. At each pause it weighs the user's words against the bot's last line.
This one changes behavior on upgrade. A turn analyzer next to a speech timeout now ends the turn only when both agree. Wake phrase bots change too.
When the user talks again after the answer has started, that answer is cancelled. And Whisper-style services stop ending a turn on earlier words.
Services.
AssemblyAI's streaming text-to-speech is now a Pipecat service. An interruption cancels speech without reconnecting, and an EU host keeps processing in Europe.
Interrupt a bot on OpenAI's Responses WebSocket and the next reply starts sooner. The interrupted response finishes on its own lane, and the new request still sends only the new messages.
Amazon Nova models on Bedrock wrote their reasoning into the answer, so voice bots spoke it. That reasoning now arrives as thought frames, and Claude on Bedrock keeps its reasoning across tool calls.
I'd try expressive conversation first on a bot that reads out order numbers, with the slow for details preset, and keep it on a branch while it's alpha. The upgrade steps are in the Pipecat v1.15.0 release notes.
