MOSS-VL
A model is watching a cat. One instruction: say Great the moment the cat touches the carrot, nothing otherwise. The cat touches the carrot. The model says Great. Silence in between, every time. This is MOSS-VL, from a lab called OpenMOSS, and anyone can download the model, all twenty-two gigabytes.
One instruction at the start, and MOSS-VL runs unattended from there
The whole instruction is one sentence: watch in real time, and say Great whenever the cat touches the carrot. After that the user is silent. The video arrives frame by frame, and the model reads along and writes as it goes. The weights have been public since the fourteenth of July. What is new is the report. The team published it on the fifteenth of August with this model's first scores.
Nearly every moment, the right call is to keep watching and say nothing
Answering a question is the easy half. A model watching a live feed has to make a second call, over and over: is this the moment to speak? For almost every frame, the answer is no. Timing is its own skill, separate from answering. So how do you teach a model to stay quiet?
Staying quiet is a word the model writes to itself
After every frame of video, the model makes one choice, and it comes out as a word. That word is either stay quiet, or speak. The team added two words to the model's vocabulary for exactly that job. The same weights that write the reply make that call. The model picks that word the way the model picks any other word, by taking whichever word scores highest. Silence is one of the words the model can pick. When the model picks speak, the reply pays out across the frames that follow, while the video keeps arriving.
The training data on its own teaches the model to say nothing at all
In the training data, the moments where staying quiet is correct swamp the moments where speaking is. In one training example in twenty, the thing the model is watching for never happens, and staying quiet the whole way through is the right answer. Trained on that mix, the model finds the winning strategy: say nothing, ever. That strategy is right almost every time, and it is useless. So the team weighted the rare choice up, until speaking and staying quiet pulled equally hard.
MOSS-VL scores 66.0 on a surveillance alert test, against 37.5 for the next best model
The team put MOSS-VL on four public tests where video arrives live, and ran all four itself. On three of the four, MOSS-VL takes the best average score. The widest margin in the report is on a surveillance test: the model watches for a dangerous situation and raises the alarm itself, at the right moment. There, MOSS-VL scores sixty-six point zero. The next best score there is thirty-seven point five, and two models are tied there.
MOSS-VL wins on when to speak and loses on what is in the scene
MOSS-VL does not win everything. On the fourth test MOSS-VL comes second on the average, sixty-nine point seven against seventy-one point one for AURA, the strongest model the team tested against. Across the whole table, seven scores go the other way, five of them to AURA. The report says where: wherever a test comes down to answering a question about the scene in front of the model, AURA keeps the edge. The skill this training added is timing.
MOSS-VL is built to keep watching while it speaks, and that behaviour is still unmeasured
MOSS-VL was designed to do one more thing that the team could not score. The model is built to keep watching while it speaks, so it can cut a reply short the moment the scene turns over. The team says the public tests stop one step short of measuring that, and that building the missing test is next on its list.
Timing is a skill a model has to be taught on its own
So that is a model watching a cat, quiet until the cat touches the carrot. The team taught this model when to speak by making silence a word the model writes to itself. Twenty-two gigabytes, Apache licensed, on Hugging Face.
