Accelerate Bench: Claude Sonnet 5.5 leads 16 agents at making programs faster
2 days ago
Accelerate Bench gives 16 coding agents 32 working scientific programs, like GROMACS and OpenFOAM, and eight hours each to make them faster. Claude Sonnet 5.5 in Claude Code cuts about two-thirds of the running time on average and leads the field; most agents keep their programs correct, so speed decides the ranking. One run per agent, CPU tasks only.
Links
Ask about this presentation
Chapters
Show transcriptHide transcript
Claude Sonnet 5.5 in Claude Code makes real scientific programs faster than any of the 16 coding agents on Accelerate Bench. A team at Tencent built the benchmark with HKUST, Tsinghua and Peking University.
Each of the 32 tasks gives an agent a working program, like GROMACS or OpenFOAM, and eight hours. It can change code, build flags and thread settings, and a hidden verifier times the result on unseen inputs.
Across all 32 tasks, Claude Sonnet 5.5 cuts about two-thirds of the running time, and Claude Opus 5.5 comes second. Below those two, one run per agent can't rank models a few points apart.
Claude Opus 5.5 lost three tasks when safety classifiers flagged them as cybersecurity or biology work, and the test setup blocked Claude Code's fallback to an older Opus. On its other 29 tasks, it nearly matches Sonnet.
Of 512 submitted programs, 472 pass every correctness check. Speed separates the agents. Both Claude agents double the speed on 22 or more tasks, and the other fourteen on 8 to 17.
In 476 runs from five agents on the benchmark's first version, changes to the computation itself show up in 225. Compiler and build tuning shows up in only 40, yet the build can decide the result.
In one GROMACS run, the agent changed only build and runtime settings. Replayed four ways, that same source ran anywhere from under a third of the original speed to twice as fast.
In the quantum simulator QiboTN, deferring the tensor library imports did almost nothing alone. Once circuits also went to a fast exact simulator, the same change made the program more than six times faster.
In one Palabos run, the default thread count saw all 192 host CPUs and took 11 minutes. Four threads took a second and a half. About half of all runs hit a dependency like this one.
Each agent ran once, on CPU tasks shrunk to fit one machine, and repeat runs moved scores by two to four points. And each score belongs to a model and its harness together, Codex or Claude Code.
If I had a slow program, I'd hand it to Claude Sonnet 5.5 in Claude Code, as long as it times the whole run and I check its build and thread settings. That comes from one run per agent, on CPU only. The tasks are on GitHub as Accelerate Bench.
Up next
2:34Jeveration makes a 512-pixel image in 78 ms with one pass of Qwen3.5
@ai-papers
21 hours ago
2:49NestOne cuts a Qwen3.5 decision model to 24 layers at nearly full accuracy
@ai-papers
1 week ago
2:33MME-VLA robot memory counts a pause as work done and stops early
@ai-papers
1 week ago
2:16GPT-6 Astra fixed a small Qwen’s math by editing 1,023 of its weights
@ai-papers
1 week ago
2:42LoopVQ cuts looped models’ KV cache to half a bit and lifts Ouro’s throughput up to 6×
@ai-papers
2 weeks ago
2:35CUA-Master-27B is an open computer-use agent that trails only Claude Opus 5 and GPT-5.6-Sol on Agents' Last Exam
@ai-papers
2 weeks ago
2:37CaPtain trains Qwen3.5-4B to control a robot in code and beats several frontier-model agents
@ai-papers
2 weeks ago
2:45Trajectory2Gym data lifts a 4B Qwen3 agent past the stock 30B model on Tau2
@ai-papers
3 weeks ago
2:46“Wait” in reasoning models comes from training data, and RL almost never invents it
@ai-papers
3 weeks ago
2:08Looking Ahead makes Wan video models more physically plausible, with generation unchanged
@ai-papers
4 weeks ago
2:30OpenVLHarness gives vision models tools and a memory, and seven models each gain about 10 points
@ai-papers
1 month ago
2:413DCodeVerse makes the same model write better 3D code than Claude Code, Codex or Gemini CLI
@ai-papers
1 month ago














