llama.cpp v1.0.0 runs MoE models too big for your GPU up to 2× faster

llama.cpp 1.0 runs mixture-of-experts models that don't fit on your GPU up to twice as fast. With Qwen 3.8 Flash Next and a 6 gigabyte expert cache, generation climbs from about 24 to 48 tokens a second.

Slide text

llama.cpp

v1.0.0 · November 2026

Qwen3.8-Flash-Next IQ4_NL · 64,800 MiB of experts in RAM · 6,464 MiB GPU cache · generated tokens per second

Every miss uploaded

24.2

CPU takes 95% of misses

47.6

llama.cpp v1.0.0 release notes · #30966 · docs/moe-cache.md

01 Experts on the CPU

First, how that works.

Slide text

llama.cpp

v1.0.0 · November 2026

01

Experts on the CPU

llama.cpp v1.0.0 release notes

01 Experts on the CPU

Until now, every expert missing from the GPU cache was uploaded. Now the CPU computes some of those misses straight from RAM. If your experts are pinned in memory, uploading often stays faster.

Slide text

llama.cpp

v1.0.0 · November 2026

• Cache hits run on the GPU
• Cache misses a share runs on the CPU, straight from RAM

Pinned experts upload at full link speed, so the CPU share often doesn’t help

docs/moe-cache.md · llama.cpp v1.0.0 release notes · #30966

01 Experts on the CPU

The right cache size and CPU share depend on your RAM, your PCIe link and the model. A new script runs your server six times, in a few minutes, and prints the settings to use.

Slide text

llama.cpp

v1.0.0 · November 2026

$ python3 scripts/moe-cache-tune.py -- \
    llama-server -m model.gguf -cmoe -t 8 -c 32768
6 runs of llama-server, the first generates 4096 tokens, the others 1024

  cache MiB  share   t/s  hit rate
       6464      0  24.2     73.3%
       6464   0.95  47.6     61.5%
      19392      0  65.5     93.5%
      19392    0.5  66.4     91.9%

docs/moe-cache.md · llama.cpp v1.0.0 release notes · #30966

01 Experts on the CPU

Then, the version number itself.

Slide text

llama.cpp

v1.0.0 · November 2026

02

llama.cpp 1.0

llama.cpp v1.0.0 release notes

02 llama.cpp 1.0

llama.cpp and its tensor library, ggml, both reach 1.0. Each release now carries a matching ggml release, so Homebrew and Debian packages stay compatible. The llama.app installer and Winget still ship nightly builds.

Slide text

llama.cpp

v1.0.0 · November 2026

Each release tag carries the exact ggml release, so Homebrew and Debian link a matching copy

Still on nightly builds the llama.app installer and Winget

docs/release.md · ggml discussion #1579 · ggml v1.0.0

02 llama.cpp 1.0

To reach 1.0, the project deleted everything it had marked deprecated. That is 35 functions in llama.h, old ggml operations and the defrag threshold option, so apps built on the C library and scripts that pass that option need changes.

Slide text

llama.cpp

v1.0.0 · November 2026

• llama.h 35 deprecated functions
• ggml.h deprecated ops and functions
• --defrag-thold the option is gone

llama.cpp v1.0.0 release notes · #30565 #29801 #32184

02 llama.cpp 1.0

Next, speech.

Slide text

llama.cpp

v1.0.0 · November 2026

03

Speech

llama.cpp v1.0.0 release notes

03 Speech

llama-server adds an OpenAI-compatible speech endpoint. An app that already calls OpenAI for text to speech can point at your own machine instead, and a local TTS model does the talking.

Slide text

llama.cpp

v1.0.0 · November 2026

llama-server answers OpenAI’s text-to-speech API

POST /v1/audio/speech OpenAI-compatible, served by your local TTS model

llama.cpp v1.0.0 release notes · #31763

03 Speech

New models you can run.

Slide text

llama.cpp

v1.0.0 · November 2026

04

New models

llama.cpp v1.0.0 release notes

04 New models

Kimi K3.1 now runs locally, and it reads images too. Ripple, a dense 65 billion parameter model from ricursiveAI, arrives as well, and so does Fish Audio's Drama-3 voice model.

Slide text

llama.cpp

v1.0.0 · November 2026

• Kimi K3.1 text and vision
• Ripple-65B-Preview dense 65B from ricursiveAI, Apache-2.0
• Drama-3 text to speech from Fish Audio

llama.cpp v1.0.0 release notes · #31779 #31980 #31692 #31613

04 New models

Then, speed on more GPUs.

Slide text

llama.cpp

v1.0.0 · November 2026

05

Faster GPUs

llama.cpp v1.0.0 release notes

05 Faster GPUs

A new flash attention kernel speeds up prompts on Intel GPUs through Vulkan. On an Arc Pro B70, an 8,000-token prompt runs almost three times faster. The Arc A770 stays on the old path for now.

Slide text

llama.cpp

v1.0.0 · November 2026

Intel Arc GPUs process prompts almost 3× faster

Qwen3 8B Q4_K_M · 8,192-token prompt · Arc Pro B70, Windows, Vulkan · prompt tokens per second

Before

1,065

v1.0.0

2,956

Arc A770 stays on the old path for now

llama.cpp v1.0.0 release notes · #29357

05 Faster GPUs

The 8-bit FP8 format now runs on NVIDIA GPUs through CUDA, and the KV cache can be stored in FP8 too. FP8 kernels for mixture-of-experts layers and for prompt processing come in later pull requests.

Slide text

llama.cpp

v1.0.0 · November 2026

FP8 on CUDA FP8 KV cache

Still to come FP8 kernels for MoE layers and for prefill

llama.cpp v1.0.0 release notes · #28413 #28392

05 Faster GPUs

A new CUDA kernel handles the small batches that speculative decoding checks. With Qwen 3.8 27B and a draft model, an RTX 3090 Ti generates about a third faster, and newer cards gain a little less.

Slide text

llama.cpp

v1.0.0 · November 2026

• RTX 3090 Ti about a third faster
• RTX 5090 about a fifth faster
• Supported Ampere through DGX Spark

llama.cpp v1.0.0 release notes · #31533

05 Faster GPUs

Next, loading across machines.

Slide text

llama.cpp

v1.0.0 · November 2026

06

Faster loading

llama.cpp v1.0.0 release notes

06 Faster loading

Loading a model across machines over RPC got much faster. DeepSeek V4 Flash on two DGX Sparks loads in about a minute instead of seven and a half. The protocol changed, so older clients and servers are now refused.

Slide text

llama.cpp

v1.0.0 · November 2026

Models load up to 8× faster across machines

DeepSeek-V4-Flash · 145.6 GiB · two DGX Sparks over RPC · seconds to load, lower is better

Before

450 s

v1.0.0

57 s

Upgrade both ends older RPC clients and servers are refused

llama.cpp v1.0.0 release notes · #29413

06 Faster loading

Then, the server's defaults.

Slide text

llama.cpp

v1.0.0 · November 2026

07

A stricter server

llama.cpp v1.0.0 release notes

07 A stricter server

An exposed llama-server now refuses tools without an API key or an explicit opt-in, and keeps its proxy and media links off local addresses. Media on your home network needs an opt-in, and the proxy can't reach a local MCP server at all.

Slide text

llama.cpp

v1.0.0 · November 2026

llama-server locks down tools and local addresses

• Tools and MCP need an API key or an opt-in on a network address
• Local addresses blocked for the proxy and media URLs
• Hugging Face token sent only to huggingface.co

Media on your LAN needs an opt-in · the proxy can’t reach a local MCP server

llama.cpp v1.0.0 release notes · #32113 #32115 #31730

07 A stricter server

And the web interface.

Slide text

llama.cpp

v1.0.0 · November 2026

08

The web UI

llama.cpp v1.0.0 release notes

08 The web UI

In the web interface you can now gather chats into named groups in the sidebar. The message box expands into a large editor with a live markdown preview, and on phones the app feels native.

Slide text

llama.cpp

v1.0.0 · November 2026

Gather chats into named groups

• Groups a sidebar section each
• Editor a large one, with markdown preview
• Phones a native feel

llama.cpp v1.0.0 release notes · #32012 #31973 #31949

08 The web UI

If you keep a mixture-of-experts model's experts in RAM, update and run the tuning script first. If you build on the C library, the 1.0 release notes and the prune pull request list everything that was removed.

Slide text

llama.cpp

v1.0.0 · November 2026

Building on llama.h? The llama.cpp v1.0.0 release notes and the llama : prune pull request list what was removed

github.com/ggml-org/llama.cpp · releases/tag/v1.0.0

llama.cpp v1.0.0