Transformers adds Reflection AI’s Beam and loads models on PyTorch 2.7 again

Transformers now runs Beam, the first open-weight model from Reflection AI. It's a mixture of experts with 501 billion parameters, 23 billion of them active per token, built for coding, reasoning and agent work.

Slide text

Transformers

01 02 03 New model

huggingface/transformers · release v5.20.1 · PR #49664

Transformers now runs Beam, Reflection AI’s open-weight model

501B parameters 23B active per token Coding and agents

The BF16 checkpoint is about a terabyte. The examples shard it across eight GPUs with expert parallelism, which puts whole experts on each device. That beats device_map auto, which can run out of memory while loading.

torchrun --nproc-per-node 8 script.py
DistributedConfig(tp_size=8, ep_size=8)

Beam reasons before every answer, and a reasoning effort setting in the chat template controls how much. It runs from minimal up to max, the default. If the token limit runs out mid-reasoning, you get thinking and no answer.

On PyTorch 2.7, every from_pretrained call in 5.20.0 failed with this missing module error. Loading works again, and so does tensor-parallel loading on 2.6 and 2.7. Distributed checkpointing now needs PyTorch 2.8.

ModuleNotFoundError: No module named
'torch.distributed.checkpoint.hf_storage'

Since 5.20.0, generate crashed for encoder-decoder models and some vision-language models whenever the input IDs sat on a different device from the model. It was rebuilding their encoder outputs as plain dictionaries. It keeps their type now.

Eight vision-language models, including Qwen3.5 and Muse Spark, declared text as their only input, so generate treated them as text-only. They take the multimodal path now. Aria, Inkling and PaddleOCR-VL have the same bug and still need their own fix.

With expert parallelism under FSDP2, and an expert group smaller than the FSDP group, bfloat16 expert gradients came back as zero, so the experts never learned. They get real gradients now.

HunYuan-VL gives each beam its own image in batched generation. DeepSeek V4 with the DeepGEMM Mega MoE kernel runs under expert parallelism instead of failing on its first forward pass. And Gemma 4 audio stops forcing a GPU sync on every mask.

The full list of fixes is in the release notes for 5.20.1 and 5.20.2 on GitHub. Which PyTorch version is your training setup pinned to?