Transformers v5.19.0: EmbeddingGemma 2 and token-dispatch expert parallelism

Transformers v5.19.0 adds EmbeddingGemma 2. It's a multimodal embedding model from Google, built on the Gemma 4 architecture.

Slide text

01 02 03 04 05 06 EmbeddingGemma 2

huggingface/transformers · GitHub release v5.19.0 · Transformers docs · EmbeddingGemma 2

Transformers v5.19.0 adds EmbeddingGemma 2

Multimodal embeddings From Google Gemma 4 architecture

It encodes text, images, audio and video, one at a time or mixed in a single input, into one shared space. So a text query and a photo can be compared directly.

Slide text

Transformers v5.19.0

01 02 03 04 05 06 EmbeddingGemma 2

huggingface/transformers · GitHub release v5.19.0

Text, images, audio and video land in one vector space

768 dimensions, for retrieval, similarity, clustering and classification across media

Embeddings come out at 768 dimensions. Because it's trained with Matryoshka Representation Learning, you can keep just the first 512, 256 or 128, and the docs report minimal quality loss.

Embedding only text? You can disable the vision or audio tower at load time to save memory. The token budget per image or video frame is configurable as well.

Expert parallelism.

Expert parallelism gets a second mode, token dispatch. Each rank keeps its own tokens and sends only routed tokens to the experts' owners, so expert and tensor parallel sizes become independent.

Token dispatch is now the default for Qwen3 MoE and Mellum. The older masked mode, where every rank sees every token, is still there if your plan selects it.

The Trainer now works with expert parallelism. Tensor and expert parallel plans are decoupled, each with its own device mesh. And the docs now cover tensor parallelism for PEFT adapters.

Continuous batching.

The standard SDPA and flash attention functions now support continuous batching themselves. So the paged prefix on those two is deprecated. Eager attention still needs its paged prefix.

Inside continuous batching, the index path and block table path now update the cache in one fused call. Custom code that called those update paths separately will need changes.

The cache.

DynamicCache and StaticCache used to reuse one set of settings for every layer. Now each layer reads its own sliding window, chunk size, conv states and head counts, which suits models with mixed layers.

Generate used to edit the cache config you passed, so a second model could inherit the first one's settings. And beam search now reorders quantized cache states correctly.

Outputs that change.

Ask for router logits, and every mixture of experts model whose router computes them now returns them, the way Qwen3 MoE already did. With the flag off, the defaults are unchanged.

OWLv2 image-guided detection now picks the query box with the highest objectness, as the original OWLv2 notebook does. Here, the old pick boxed the whole lemon photo. Expect detections to differ from earlier releases.

Slide text

Transformers v5.19.0

01 02 03 04 05 06 Changed outputs

huggingface/transformers · pull request #49200

OWLv2 picks its image query by objectness

Top: the old pick boxes the whole image. Bottom: the new pick boxes the lemon.

The language modeling data collator no longer ignores a seed of zero. Deprecated pipeline tasks now redirect. And Trainer subclasses can declare whether their loss is already scaled for gradient accumulation.

The full list of changes is in the Transformers v5.19.0 release notes on GitHub. That's Transformers 5.19: EmbeddingGemma 2, and token dispatch for expert parallelism.