huggingface_hub's HfFileSystem now downloads a 2 GB bucket file in seconds

HfFileSystem now downloads Xet files through hf_xet, which writes straight to disk. A 2 gigabyte parquet file in a bucket went from two minutes to about seven seconds.

Slide text

01 02 03 04 05 06 Faster downloads

huggingface/huggingface_hub · release v2.1.0 · PR #4990

HfFileSystem downloads a 2 GB file in seconds

get_file · 2 GB parquet in a bucket 120 s 7 s

The biggest winner is the datasets library loading from a bucket without streaming. That went from about 130 seconds to 17. Files that aren't on Xet, or machines without hf_xet, still use plain HTTP.

Jobs got four new controls.

A job can now retry itself when it fails. You set the number of attempts when you create it, from Python or the hf command line.

You can also rerun an earlier job. It starts fresh from the old job's saved spec, secrets and hardware settings included.

Scheduled jobs can be rescheduled. You change the cron expression or preset on the existing job, without deleting and recreating it.

Jobs can also open ports. A port can require a token, or be public. You can open one at launch, or on a job that's already running.

Next, ZeroGPU quota.

A new method reports how much ZeroGPU time you have left and when it resets. Apps, MCP servers and agents on ZeroGPU Spaces can warn users before the daily limit.

The command line shows the same numbers, and suggests buying credits when less than a tenth is left. The patch release renamed this command, and the first name was removed without an alias.

The Inference Endpoints catalog moved to a new API.

The model catalog now returns each model with its tested deployment recipes. A recipe pairs hardware with a serving engine. The server filters by accelerator, engine, license, task or search.

You can deploy one exact recipe, or a specific GGUF file. The listing now returns objects instead of model names, and the command line prints a table.

Two changes break old behavior on purpose.

The client now refuses a pooled sandbox without a valid scoped token, instead of falling back to the host's credential. New clients can't reach older pool hosts, so upgrade or recycle them.

Bucket include and exclude patterns used to ignore case on Windows only. So one sync could pick different files on each system, which is risky when it deletes. Now case always counts.

Some fixes.

Copying into a bucket no longer mistakes a similar file name for a folder. Paginated listings retry their first page. And a single model card tag stays one tag.

Listings escape control characters, so a file name can't rewrite your terminal. A job secret that isn't set locally now raises an error. And who am I tells you how to log in.

Every change is in the release notes on GitHub, and the Jobs and Inference Endpoints guides cover the new options.