744B parameters, streaming off your SSD one word at a time
A seven hundred and forty-four billion parameter model, answering on one ordinary computer. colibrì does it by never loading the whole thing. The model still takes about three hundred and eighty-four gigabytes on your drive. It runs on twenty-five gigabytes of RAM. That gap is the whole story.
colibrì is not a model. It is the engine that runs one.
colibrì is not a model. It is the engine that runs one. You point it at GLM-5.2, the open weights model from Z.ai, and it answers. Here someone asks it how a hummingbird hovers. colibrì is Italian for hummingbird, so the demo answers the question its own name asks. And the machine on your desk is closer to this than you assumed.
Why streaming works: only ~5% of the model computes each token
This model never uses all of itself. For every word it writes, the router picks eight experts out of two hundred and fifty-six in each layer. Forty billion parameters do the work. Seven hundred and four billion sit still. And only about eleven gigabytes of that changes from one word to the next. That is all colibrì has to move.
Where the pieces live
So colibrì splits the model by how often you need each piece. The dense core runs on every word. At four bits that is nine point nine gigabytes, and it stays in RAM. The routed experts are everything else. Nineteen thousand of them, about nineteen megabytes apiece, sitting on your drive until they are called. It works like a game streaming in the level you are walking into.
The per-token path:route → union → place → overlap → learn
Every layer of every word runs the same five steps. Route. The router scores all two hundred and fifty-six experts and keeps the top eight. Union. If two words want the same expert, it gets read once. Place. Whatever is already in memory runs now. Overlap. Those experts compute while the drive delivers the missing ones, and it looks a layer ahead. Learn. The counters update and the hot set re-ranks. And here is the line the whole project rests on. Placement decides speed only. Precision and routing are never quietly changed.
It learns what you use
And this part has real personality. colibrì writes down which experts your work reaches for. Then it pins those into whatever spare memory you have. The same machine gets faster the longer you use it. One person running a Framework 13 reported a forty-six gigabyte pin catching sixty-six percent of the experts it needed. Another watched an M5 Max climb from one point one one to one point eight three tokens a second inside a single run.
So why is it slow?
So why is it slow? On the cheapest machine one word takes ten to twenty seconds, and the project's own docs say it plainly. This is not fast. But look at what is slow. Someone swapped only the drive in a big desktop. Same chip, same model, an older drive out and a PCIe five drive in. Throughput went from zero point one to zero point two eight tokens a second. And the disk fell from sixty-six percent of the time down to thirty-two. The math is the bottleneck now. Five point eight times the bandwidth bought two point nine times the tokens. The ceiling is your drive, not the idea.
The whole engine is one C file
The whole engine is one C file. The Linux build that shipped this morning is three hundred and twenty thousand bytes. The model it drives is about three hundred and eighty-four gigabytes. That model is roughly one point two million times the size of the program running it. Version one point oh landed today, with builds for Linux, macOS and Windows. Apache two for the engine. The weights are Z.ai's, under MIT.
It was never a size problem. It was a placement problem.
So if you assumed the biggest models need a data centre, this one says the problem was placement, not size. Seven hundred and forty-four billion parameters, on one machine. The link is on screen. New repo tomorrow.
