Which agents work with it?

Any agent that reads skills from a directory. Claude Code, Codex and Gemini CLI read it natively; Cursor and VS Code connect over MCP.

The voice runs on your own machine, so there is nothing to meter. The first run downloads it, about 340 MB.

Yes. The skill is MIT-licensed and the authoring format is public.

Create

MME-VLA robot memory counts a pause as work done and stops early

3 days ago

@ai-papers Subscribe

MME-VLA, a memory-augmented robot policy on RoboMME, counts a pause as work done: two 5-second pauses cut its success on a counting task from 47 to 16 of 50, because its memory is filled by time. Indexing memory by events at test time wins back half or more of the loss. Paper: https://arxiv.org/abs/2611.14488 Code: https://github.com/bennidict23/memory-clock

Ask about this presentation

Show transcript

Pause MME-VLA, a robot policy with memory, for five seconds, twice, and it decides the job is further along than it is. It presses the finish button early. That failure is phantom progress, found by a Nanyang Technological University team.
Slide text
MME-VLA robot memory counts a pause as work done and stops early
Chubin Zhang and Bo An · Nanyang Technological University
Zhang & An · Is Your Robot’s Memory Just a Clock? · arXiv 2611.14488v1 · submitted 10 Nov 2026
The task: pick up a cube and place it three times, then press a button. With two pauses, the robot presses after two repeats, at about the moment it would have pressed with no pauses.
Slide text
01 02 03 What breaks
Two pauses, and the robot presses after 2 of its 3 repeats

Zhang & An · Is Your Robot’s Memory Just a Clock? · arXiv 2611.14488v1 · Figure 1 · RoboMME PickXtimes · released FrameSamp-Modul policy
Without pauses, the released policy finishes the counting task in nearly every run. With them, it finishes about a third. Keep the same pauses out of its memory, and success comes back. The memory is what breaks.
Slide text
01 02 03 What breaks
Pauses cut success to about a third, and hiding them from memory restores it

No pause
47 of 50

Pauses in memory
16 of 50

Pauses hidden
45 of 50

Two 5-second pauses · PickXtimes · SwingXtimes falls from 44 to 10 · almost every failure is an early press
Zhang & An · Is Your Robot’s Memory Just a Clock? · arXiv 2611.14488v1 · Figure 2, §4 · 50 paired episodes per condition
Why would it confuse waiting with working? Demonstrations move at a steady pace. From the clock alone, you can guess how many repeats are done eight or nine times in ten, in simulation and in real robot data.
Slide text
01 02 03 Why
In demonstrations, elapsed time alone tells you how many repeats are done

89% right on PickXtimes, guessing the count from the time step alone
92%+ right on three-repeat demonstrations from two real teleoperation datasets
Zhang & An · Is Your Robot’s Memory Just a Clock? · arXiv 2611.14488v1 · Figure 3, §5.1 · austin_sailor and stanford_hydra from Open X-Embodiment
So the robot reads the clock. With no pause at all, making its time stamps run fast cuts success from nearly every run to about a third, all early presses. A slow clock makes it repeat too many times.
Slide text
01 02 03 Why
Speed up the time stamps alone, and the robot quits early

Zhang & An · Is Your Robot’s Memory Just a Clock? · arXiv 2611.14488v1 · Figure 4, §5.2 · no pause injected, frames unchanged · PickXtimes
Deleting the time stamps doesn't cure it. The memory keeps thirty-two frames spread evenly over the episode, so a pause still takes up slots. A model trained with no clock still failed most runs under pauses.
Slide text
01 02 03 Why
Without time stamps, pause frames still fill the memory

32 frames sampled evenly over the episode · two 5-second pauses took 10 of them in this run

21 and 14 successes of 50 under pauses for NoClock, a model trained with no time encoding
Zhang & An · Is Your Robot’s Memory Just a Clock? · arXiv 2611.14488v1 · Figure 6, §5.3 · NoClock and Clock trained from the π0.5 base for 30k steps
Every released Robo MME memory that samples frames by time collapses. One that keeps changed pixels shrugs off pauses but not a fidgeting wrist. Memories that log sub-goals hold up, though they count less well.
Slide text
01 02 03 Why
Memories that count by time collapse, and memories that count sub-goals hold

Frames sampled by time all three released FrameSamp variants fall to between 10 and 20 of 50

Tokens kept by pixel change TokenDrop barely notices a pause, but a wrist fidget still hurts it

Progress logged by sub-goal a keyframe memory and a text memory are unaffected, and count worse to begin with

Zhang & An · Is Your Robot’s Memory Just a Clock? · arXiv 2611.14488v1 · Table 2, §5.4 · RoboMME’s released memory policies, 50 paired episodes
Training with fake pauses didn't fix it, and the robot got worse at counting. Keeping only frames where the world actually changed did, with no retraining. It at least doubled success under every kind of pause.
Slide text
01 02 03 The fix
Training on pauses fails, and indexing memory by events wins back half or more

Zhang & An · Is Your Robot’s Memory Just a Clock? · arXiv 2611.14488v1 · Figure 8, §6 · event index applied at test time, no retraining
All of it runs in simulation, with scripted pauses, on one family of policies. The event detector reads simulator signals, can mistake a real repeat for nothing, and is wrong for tasks where timing matters.
Slide text
01 02 03 The fix
Simulation only, one policy family, and a detector that reads the simulator

Simulated and scripted every pause is injected, so how often real robots stall is not measured

One family dissected MME-VLA on π0.5; two other policies probed without dissection

Event detector uses simulator signals, can mistake a real repeat for nothing, and is wrong where duration is the task

Zhang & An · Is Your Robot’s Memory Just a Clock? · arXiv 2611.14488v1 · §6, §7 Limitations · most models trained with one seed
If I were giving a robot memory, I'd advance it on events and pass time in as a separate input. The paper is on arXiv, and the test harness is on GitHub.
Slide text
01 02 03 The fix
Is Your Robot’s Memory Just a Clock?

Paper arxiv.org/abs/2611.14488
Code github.com/bennidict23/memory-clock

Building robot memory: count events, keep the clock as an input.
Zhang & An · Is Your Robot’s Memory Just a Clock? · arXiv 2611.14488v1 · code MIT-licensed · null-event tests, clock rescaling, event index

Up next