Transformers v5.18.0: streaming speaker diarization, GGUF mixture-of-experts, FSDP2 with expert parallelism

Transformers 5.18 adds Nemotron 3 Diarization, an open-weight model that works out who spoke when. It tracks up to eight speakers, labeled in the order they first speak.

Slide text

01 02 03 04 New models

huggingface/transformers · GitHub release v5.18.0

Nemotron 3 Diarization Up to 8 speakers Open weights

One checkpoint streams with a buffer as short as 80 milliseconds, or takes about 30 seconds, like an offline run. Chunked inference puts no limit on audio length.

Slide text

Transformers v5.18.0

01 02 03 04 New models

huggingface/transformers · GitHub release v5.18.0

One checkpoint streams or runs offline

NVIDIA's NemotronH Omni pairs a hybrid Mamba-Transformer language model with a RADIO vision encoder and an optional Parakeet sound encoder, so one model reasons across text, images, video and audio.

NAVER's HyperCLOVAX Vision V2 takes text, images and video, and can reason step by step with built-in thinking tokens before it answers.

GTE is a BERT-style encoder that uses RoPE in place of absolute position embeddings. It's the architecture behind Alibaba's GTE checkpoints and Snowflake's Arctic Embed M version 2.

Running models.

GGUF loading now handles mixture-of-experts models, though only Qwen 3.5 so far. On Apple Silicon, the 35-billion-parameter Qwen 3.5 generated text as fast as it does in llama.cpp.

The transformers chat command can now load a GGUF file from a model repo. The serve command already could; chat took only a plain model name.

When a requested flash kernel won't run, Transformers picks one that runs on your GPU. Apple GPUs get FP8 kernels with PyTorch 2.15. MUSA GPUs get Flash Attention 2.

The ExecuTorch exporter gains an MLX backend, for inference on Apple Silicon GPUs. It exports with static or dynamic shapes, It rejects the static cache and points you to the dynamic cache.

Training at scale.

Expert parallelism shards only the experts, so every GPU still holds the whole dense part. The Trainer can now shard that too, with FSDP2 on a two-dimensional mesh. Before, that combination raised an error.

Inference can now combine pipeline and tensor parallelism in one run. And the PEFT integration now works with DTensor-based tensor parallelism.

Before you upgrade.

Transformers now allows version 2 of huggingface_hub. Its breaking changes don't affect Transformers, and the oldest supported version stays at 1.31.

Loading an RT-DETR or SEW-D base model left every weight random, with only a warning. PEFT adapters loaded through AutoModel dropped extra saved modules like a classifier head. The UMT5 decoder could see future tokens.

DETR-family preprocessing runs up to a third faster on GPU, and RF-DETR resizes without antialiasing. LLaVA-OneVision returns video features ready to merge with text. gpt-oss on ROCm moves to the aiter kernel.

The use_mamba_kernels flag has done nothing since 5.15, and now warns when set to false. Every other change is in the Release 5.18 notes on GitHub.