Muse Glimmer 30B
Ask an AI a question and it types the answer one word at a time. Meta just released Muse Glimmer, an open model that gets around that by guessing ahead.
Meta squeezed a 30B model down to fit a 24 GB card
to about 4 bits
Muse Glimmer is Meta's first open weights language model since April of last year. Its thirty billion parameters, the numbers it learned in training, would need over fifty-five gigabytes of memory at full size. That is more than any graphics card a person can buy. So Meta squeezed each number to about four bits, which puts the model under twenty gigabytes. On a twenty-four gigabyte card that leaves room for its working memory, the part that reads images, and a second model that guesses.
It wins at using tools and loses at driving a desktop
Artificial Analysis rates it 35 on its own index · Qwen 38 · Gemma 30
Meta ran Muse Glimmer against two rival open models its own size, Google's Gemma four and Alibaba's Qwen three point six. Across twenty-two tests, Muse Glimmer wins twelve and loses ten. The wins cluster in one place: using outside tools, searching, and long documents. The losses cluster in another, and both go to Qwen: driving a desktop, and working at a command line. That is Meta's own table, and Meta gave each rival its best published score. Artificial Analysis, the outside firm that measured eight of those rows, puts Muse Glimmer at thirty-five overall, just behind Qwen at thirty-eight.
The second model fills sixteen blanks in one pass
That second model ships in its own repository. It carries two and a half billion parameters across five layers, about a twelfth the size of the big one. It has one job: guess what the big model is about to say. And it guesses a block at a time. It lays down sixteen empty slots and fills every one of them in a single pass. Sixteen is what ships in the file, and what Meta measured.
The big model checks all sixteen guesses in one pass
To produce a single word, the graphics card pulls the whole model through its memory. Its arithmetic sits waiting for most of that trip. Checking sixteen words costs the card barely more than checking one. So the guesser hands over sixteen words at once. The big model then grades the whole block at once. It keeps the run of words it agrees with, corrects the first word it disagrees with, and carries on. Every word that survives is a word the big model approved. So you read the answer the big model would have given on its own. A bad guess costs a little time and leaves that answer alone.
The guesser reads five snapshots from inside the big model
A guesser working from the words alone has to predict the future from scratch, and it guesses badly. The researchers who designed this one trained exactly that version, a five-layer guesser with no window into the big model, and they call its speedups modest. So Muse Glimmer wires the guesser into the big model's insides. While the big model reads your question, Muse Glimmer taps a summary of what that model is thinking from five of its fifty-two layers. That summary goes into the working memory of every layer of the guesser. Guessing a block of sixteen costs about what guessing one word costs. So the guesser can afford to be deeper, and depth is what makes its guesses good enough to keep.
3.1 times on an RTX 5090
3.1× on an RTX 5090. 1.5 to 1.8× on a Mac.
A token is a chunk of text about the size of a word. On an RTX five thousand ninety, a two thousand dollar Nvidia graphics card, Muse Glimmer goes from about seventy-five tokens a second to two hundred and thirty-three. Same answer, three point one times quicker. On Apple laptops Meta measured smaller gains on different software. One point eight times on an M five Max, one and a half on an M four Max. All of it is Meta's own measurement, one request at a time. AMD has published its own hands-on numbers, up to fifty-three tokens a second on a Radeon card. The before and after pair the three point one is made of, only Meta has published.
Every word still has to pass the big model
Every word still has to pass the big model. The small one just gets there first.
So the dribble of one word at a time is a memory problem, and the fix is a guess that gets checked. Both squeezed builds and the guesser sit on Hugging Face under Apache two point oh, ungated. Meta says tuned support for running models at home lands in the coming days, so these numbers are the floor.
