CUA-Master-27B is an open computer-use agent that trails only Claude Opus 5 and GPT-5.6-Sol on Agents' Last Exam

1 hour ago

CUA-Master-27B is an open computer-use agent built on Qwen-3.5-27B that works a desktop with clicks and shell commands in the same task. On Agents' Last Exam it passes 25% of tasks, behind only Claude Opus 5 and GPT-5.6-Sol and level with Kimi K3, and it scores 80.2 on OSWorld Verified at about eight minutes a task. Long OSWorld 2.0 jobs and CAD work remain weak. From arXiv 2611.03126v1 (submitted 2 November 2026), MMLab at CUHK, Tsinghua University, HKUST (Guangzhou) and JD Explore Academy. Paper: https://arxiv.org/abs/2611.03126 Code: https://github.com/xlyu0106/CUA-Master Weights: https://huggingface.co/yxl66666/CUAMaster-27B

Ask about this presentation

Chapters

  1. 0:00CUA-Master-27B runs a computer with clicks and shell commands
  2. 0:18Only two models pass more of Agents' Last Exam
  3. 0:34One task, two ways of acting
  4. 0:46Both modes beat either alone
  5. 1:00About eight minutes a task
  6. 1:12Training tasks start as tutorial videos
  7. 1:29Rewind instead of restart
  8. 1:45Long jobs and CAD still mostly fail
  9. 2:04Rebuilding it takes a lab
  10. 2:19Where to get it
Show transcript
  1. CUA-Master-27B runs a computer for you, clicking through apps and typing shell commands in the same task. It's an open model built on Qwen 3.5, and on Agents' Last Exam only Claude Opus 5 and GPT-5.6 Sol finish more tasks.

    Slide text
    AI Papers

    CUA-Master-27B is an open computer-use agent that trails only Claude Opus 5 and GPT-5.6-Sol on Agents' Last Exam

    Xinlei Yu, Gen Li, Ji Zhao and 15 co-authors · MMLab, CUHK · Tsinghua University · HKUST (Guangzhou) · JD Explore Academy

    Three CUA-Master runs that alternate shell commands and clicks: filling a web application form, grading exam papers into a Calc spreadsheet, and checking brain-scan masks
    Yu et al. · CUA-Master · arXiv 2611.03126v1 · submitted 2 Nov 2026 · Figure 5
  2. Agents' Last Exam gives long jobs across desktop apps and the terminal, and counts a task only if every check passes. CUA-Master passes one in four, level with Kimi K3, an open model about a hundred times its size.

    Slide text
    01 02 03 What it does

    Only Claude Opus 5 and GPT-5.6-Sol pass more of Agents' Last Exam

    Four bar charts comparing CUA-Master-27B with frontier and open models on OSWorld Verified, OSWorld 2.0, WeaveBench and Agents' Last Exam
    Yu et al. · CUA-Master · arXiv 2611.03126v1 · Figure 1 · Table 1
  3. Grading a stack of exam papers, it lists the files with a shell command, reads the marked answers on screen, and scores them in Python. Then it types the totals into a Calc spreadsheet.

    Slide text
    01 02 03 What it does

    Grading exam papers, it finds files in the shell and enters scores in Calc

    31 shell commands
    20 clicks and keys

    One run of 61 steps · it switched modes 8 times

    Yu et al. · CUA-Master · arXiv 2611.03126v1 · Figure 5
  4. WeaveBench mixes web, design, data and document jobs. With only clicks or only commands, the model scores about 12 out of 100 at best. Allowed both, it scores about 44.

    Slide text
    01 02 03 What it does

    With clicks alone or the shell alone, it fails most WeaveBench tasks

    WeaveBench · 114 tasks · average score out of 100

    clicks only
    6.6
    shell only
    11.5
    both
    43.9
    Yu et al. · CUA-Master · arXiv 2611.03126v1 · Figure 6 · Section 3.3.1 · Table 6
  5. It averages about eight minutes per OSWorld Verified task, a minute quicker than Claude Opus 5. On Agents' Last Exam it needs about a third less time than the models it ties.

    Slide text
    01 02 03 What it does

    It takes about eight minutes per OSWorld Verified task

    OSWorld Verified 80.2 in 8 min · Claude Opus 5: 86.3 in 9 min
    Agents' Last Exam same pass rate as Kimi K3 and Qwen-3.8-Max, in 34–36% less time

    Served on one node of 8 H200 GPUs with vLLM · runtime depends on the serving setup

    Yu et al. · CUA-Master · arXiv 2611.03126v1 · Table 1 · Section 3.1.3 · Section 3.2.2
  6. Its training data starts from how-to videos for more than 650 apps. A model cuts each video into tasks and rebuilds the clicks, then recombines those steps into new, checkable tasks across several apps.

    Slide text
    01 02 03 How it learned

    Its training tasks start as how-to videos on YouTube, TikTok and Bilibili

    CUA-Master pipeline: tutorial videos are harvested, cut into seed tasks, rebuilt into action trajectories, broken into reusable primitives, recombined into new tasks and collected in virtual machines with snapshot rollback
    Yu et al. · CUA-Master · arXiv 2611.03126v1 · Figure 2 · Sections 2.3–2.4 · Appendix B
  7. When a practice run fails, the pipeline rewinds the virtual machine to its last good state and retries with a hint. On tasks about as hard as Agents' Last Exam, half the runs succeed, against one in seven.

    Slide text
    01 02 03 How it learned

    Rewinding a failed practice run more than triples the runs that succeed

    Synthetic tasks of 40 steps across 3 apps · share of practice runs that succeed

    restart from scratch
    14.6%
    rewind to last good snapshot
    49.8%
    Yu et al. · CUA-Master · arXiv 2611.03126v1 · Table 3 · Section 2.4.2
  8. On OSWorld 2.0, with longer tasks, it averages about 37 out of 100 and spends over an hour on each. CAD and design jobs score about 26. And stock Qwen 3.8, the same size, scores a point higher on OSWorld Verified.

    Slide text
    01 02 03 Where it stops

    Long jobs and CAD work still mostly fail

    OSWorld 2.0 36.9 average score · over an hour per task
    WeaveBench CAD and design 25.9 and 26.3
    Stock Qwen-3.8-27B, the same size 81.2 on OSWorld Verified, a point higher
    Yu et al. · CUA-Master · arXiv 2611.03126v1 · Table 1 · Table 6
  9. Rebuilding the training takes 128 H200 GPUs and a setup built for over 2,000 virtual machines at once. The data is only as good as the model labelling it, and labelling its own data adds little.

    Slide text
    01 02 03 Where it stops

    Rebuilding the training takes a lab and a strong labelling model

    128 H200 GPUs 16 nodes of 8, for fine-tuning and RL
    2,000+ virtual machines at once, to collect practice runs
    A stronger annotator gives larger gains · labelling its own data gains little
    Yu et al. · CUA-Master · arXiv 2611.03126v1 · Section 3.1.3 · Section 2.1.3 · Figure 9 · Appendix E
  10. For a computer-use agent on my own hardware, I'd try CUA-Master-27B first, in a sandbox, because it ties far larger open models on Agents' Last Exam at a size one server can run. Links are below.

    Slide text
    01 02 03 Where it stops

    The CUA-Master-27B weights and the data pipeline are released

    Paper arxiv.org/abs/2611.03126
    Code github.com/xlyu0106/CUA-Master
    Weights huggingface.co/yxl66666/CUAMaster-27B
    Yu et al. · CUA-Master · arXiv 2611.03126v1 · run it in a sandbox: its actions edit files and execute commands

What happened in AI today, in your inbox

One short email each evening. Read it and you’re caught up.

Built with ❤️ in New York