Pipecat v1.13.0 lets users send files and images to your voice bot

1 hour ago

Pipecat v1.13.0 lets users send files and images to a voice bot over RTVI. Also new: LLMWithBackend (alpha) splits talking from thinking, backchannels let a user's "mhm" pass without interrupting, Silero VAD 6.2, Azure Voice Live, classifiers that see camera frames, pipecat eval judge, credential redaction in errors, token count fixes and new default models. Release notes: https://github.com/pipecat-ai/pipecat/releases/tag/v1.13.0 CHANGELOG: https://github.com/pipecat-ai/pipecat/blob/v1.13.0/CHANGELOG.md Full diff: https://github.com/pipecat-ai/pipecat/compare/v1.12.0...v1.13.0 Docs: https://docs.pipecat.ai

Ask

Ask about this presentation

Answers are generated from this presentation.

Chapters

  1. 0:00Pipecat v1.13.0
  2. 0:13Files from the client
  3. 0:15FileResolver
  4. 0:27Rejected files
  5. 0:39Turn-taking
  6. 0:40LLMWithBackend
  7. 0:52Backchannels
  8. 1:07Silero VAD 6.2
  9. 1:20Azure Voice Live
  10. 1:33Video and classifiers
  11. 1:35Classifiers see images
  12. 1:47Video capture
  13. 2:00Evals and security
  14. 2:02pipecat eval judge
  15. 2:14Credential redaction
  16. 2:29Token counts
  17. 2:41Before you upgrade
  18. 2:43New default models
  19. 2:57Requirements
  20. 3:13Your turn
Show transcript

Pipecat v1.13.0

Pipecat01 · Files from the client

Pipecat v1.13.0 lets users send files and images to your voice bot

A new RTVI message, send-file, puts them in the LLM context

CHANGELOG.md · 1.13.0 · Added

Pipecat v1.13.0 lets a user hand your voice bot a file or an image mid-call. The new send-file message puts it in the LLM context, for Anthropic, Bedrock, Gemini and OpenAI.

Files from the client

Pipecat
01

Files from the client.

FileResolver

Pipecat01 · Files from the client

Each file is downloaded once, then served from memory

  • Bedrock and Vertex Gemini fetch s3:// and gs:// links themselves
  • Private addresses refused unless you allow them
  • 50 MB cap per download
CHANGELOG.md · 1.13.0

Pipecat fetches a file link right before each completion and caches the bytes, so a long call downloads it once. Bedrock and Vertex Gemini pull cloud storage links themselves.

Rejected files

Pipecat01 · Files from the client

A file the model rejects no longer jams the conversation

Clients send files only to bots on RTVI protocol 2.2.0

CHANGELOG.md · 1.13.0

When a provider rejects a request, Pipecat drops the oldest file added since the last good reply, and the call carries on. Clients send files only to bots on RTVI 2.2.

Turn-taking

Pipecat
02

Turn-taking.

LLMWithBackend

Pipecat02 · Turn-taking

LLMWithBackend splits talking from thinking

llm = LLMWithBackend(
    frontend=OpenAIResponsesLLMService(...),
    backend=BackendLLMWorker(
        llm=AnthropicLLMService(...),
        context=LLMContext(tools=[search_codebase,
            read_file, apply_patch, run_tests]),
    ),
)

Alpha

CHANGELOG.md · 1.13.0

LLMWithBackend, in alpha, pairs two models. The frontend keeps the conversation moving, while the backend does the deeper thinking and the tool calls and reports back as it works.

Backchannels

Pipecat02 · Turn-taking

A user’s “mhm” can now let the bot keep talking

“mhm”A backchannel, kept out of the LLM context
CHANGELOG.md · 1.13.0

A user's mhm while the bot talks can now arrive as a backchannel instead of a transcription. The bot keeps talking, the LLM context stays clean, and the bot can say its own Mm-hmm too.

Silero VAD 6.2

Pipecat02 · Turn-taking

Silero VAD 6.2 hears quiet voices and phone audio

Speech stop events can arrive slightly later than with v6

CHANGELOG.md · 1.13.0

The bundled Silero voice activity model moves to version 6.2, and catches quiet voices and low-quality phone audio more reliably. In exchange, speech stop events can land slightly later.

Azure Voice Live

Pipecat02 · Turn-taking

Azure Voice Live arrives as a speech-to-speech service

  • Function calling and caller transcription
  • Azure’s semantic VAD, multilingual too
  • Noise suppression and echo cancellation
CHANGELOG.md · 1.13.0

Azure Voice Live now runs in Pipecat as a speech-to-speech model. It calls tools, transcribes the caller and speaks in Azure's own voices. Azure OpenAI Realtime keeps its existing service.

Video and classifiers

Pipecat
03

Video and classifiers.

Classifiers see images

Pipecat03 · Video and classifiers

Classifiers can now look at a camera frame

image = await ClassifierImage.from_frame(frame)
results = await classifier.yes_no(
    "A frame from the user's camera.",
    {"person": YesNoQuestion(
        instructions="Is there a person in the frame?")},
    images=[image],
)
CHANGELOG.md · 1.13.0

Pipecat classifiers now take images. Ask whether a person is in the camera frame and get a yes or no. OpenAI Decisions and Cloudflare's Clef join as new backends.

Video capture

Pipecat03 · Video and classifiers

Camera and screen capture is now a transport setting

  • video_in_sources captures as people join
  • pipecat init offers LiveKit
  • bot-ready tells the client what the bot can do
CHANGELOG.md · 1.13.0

List the camera and screen in video_in_sources, and Daily or SmallWebRTC captures them without a connect handler. pipecat init now offers LiveKit, and bot-ready tells the client which panels to show.

Evals and security

Pipecat
04

Evals and security.

pipecat eval judge

Pipecat04 · Evals and security

pipecat eval judge re-scores a call without the bot

  • Simulation runs save each conversation
  • Reword a success criterion, get new verdicts
  • EvalConversationObserver records real calls
CHANGELOG.md · 1.13.0

Tune an eval without running the bot again. Simulation runs save their conversation for the judge to score again, and EvalConversationObserver records real calls the same way.

Credential redaction

Pipecat04 · Evals and security

Error messages now hide the keys they quote

  • Keys, tokens and URL signatures become [REDACTED]
  • OpenAI-compatible providers stop receiving your OpenAI key

Groq, DeepSeek and the rest now need their own api_key

CHANGELOG.md · 1.13.0 · Security

Errors now mask API keys, tokens and signed links. A rejected Sarvam connection used to log its key. Groq, DeepSeek and other OpenAI-compatible services stop sending your OpenAI key, and need their own.

Token counts

Pipecat04 · Evals and security

Token counts are fixed for Anthropic, Gemini and Grok

AnthropicPrompt tokens were double the real input
Gemini, GrokThinking tokens were missing from completions
CHANGELOG.md · 1.13.0 · Fixed

If you track usage, expect your numbers to move. Anthropic prompt tokens were counted twice, and Gemini, Gemini Live and Grok left thinking tokens out of completion counts.

Before you upgrade

Pipecat
05

Before you upgrade.

New default models

Pipecat05 · Before you upgrade

Gemini Live, AssemblyAI and OpenAI images get new defaults

  • Gemini Live: gemini-3.8-live
  • AssemblyAI: universal-3-6-pro
  • OpenAI images: gpt-image-2.5-flare

dall-e-3 is gone from OpenAI’s API

CHANGELOG.md · 1.13.0 · Changed

Gemini Live now defaults to Gemini 3.8 Live, and AssemblyAI to Universal 3.6 Pro. OpenAI images move to GPT Image 2.5 Flare, since OpenAI removed DALL-E 3.

Requirements

Pipecat05 · Before you upgrade

aiohttp, GStreamer and Vonage bots need a look first

  • aiohttp 3.14.0 is the new minimum
  • The gstreamer extra builds against GLib 2.80
  • Vonage now emits one mixed audio stream
CHANGELOG.md · 1.13.0 · Changed

Pipecat now needs aiohttp 3.14, so an older pin blocks the upgrade. GStreamer needs GLib 2.80, and Vonage now sends one mixed audio stream instead of one per person.

Your turn

Pipecat05 · Before you upgrade

Which LLM service does your Pipecat bot run on?

Upgrade notesPipecat v1.13.0 release notesgithub.com/pipecat-ai/pipecat/releases/tag/v1.13.0
CHANGELOG.md · 1.13.0

The steps for each change are in the Pipecat v1.13.0 release notes. So, which LLM service does your Pipecat bot run on? Tell me in the comments.