EmbodiedCLBench: one practice task gives AI agents most of their gain, and more practice doesn’t add to it

AI agents running in Claude Code, Codex, OpenClaw and Hermes get much better at unseen chores after practicing simpler ones. The first practice chore gives most of that gain, and more adds no consistent improvement. The benchmark is EmbodiedCLBench, from Peking University.

Slide text

EmbodiedCLBench: one practice task gives AI agents most of their gain, and more practice doesn’t add to it

Jie Huang, Yanan Chen, Ruixun Liu, Yiwu Zhong and colleagues · Peking University, BIGAI Beijing, University of Hong Kong

Huang et al. · EmbodiedCLBench · arXiv 2610.13021v1 · submitted 9 Oct 2026 · Figure 5

The agent works in a simulated home. It practices basic chores, like opening a drawer. Then it gets an unseen advanced chore that chains four such actions, like tidying a room before bed.

Slide text

01 02 03 The test

168 basic chores to practice on, then 100 advanced chores the agent has never seen

Huang et al. · EmbodiedCLBench · arXiv 2610.13021v1 · §3.2, §4.1 · Figure 1

At every step the agent sees a picture of the room and picks one of four moves. One heads for the goal, and some are impossible from where it stands. Advanced chores take twelve steps.

Slide text

01 02 03 The test

Each step shows a picture of the room and four possible moves

Huang et al. · EmbodiedCLBench · arXiv 2610.13021v1 · §3.3, §3.4, §4.1 · Figure 2

With no practice, the agents finish about half of the basic chores. That holds across all four harnesses and both models, Qwen3.8-Flash and GPT5.6-Terra. On the advanced chores they finish fewer than one in ten.

Now give the agent two related basic chores before the advanced one. Advanced success climbs from about one in ten to around one in three, in every harness. Codex with GPT5.6-Terra goes furthest, to about half.

One setup keeps the full log of every practice step in context. The other keeps only a short Markdown skill note the agent wrote for itself. Neither has a consistent advantage.

With practice, impossible moves drop by 13 to 20 points in every harness, and moves straight toward the goal rise by about 10. The skill note shifts these choices less than the full log.

Slide text

01 02 03 What it found

Practice mostly stops agents from choosing impossible moves

Change versus no practice, Qwen3.8-Flash · red: impossible moves · dark: moves on the shortest path

Huang et al. · EmbodiedCLBench · arXiv 2610.13021v1 · §5.3 · Figure 6

Longer chores keep the benefit. At six chained actions, an agent without practice finishes 4 percent. With practice it finishes about a quarter to a third. The full log keeps a gain of roughly 30 points.

A single practice chore lifts success from about one in ten to between 28 and 40 percent. Past that first chore, up to six, the curves wobble and none keeps climbing.

Slide text

01 02 03 What it found

The first practice chore does the work

From 1 up to 6 practice chores, the curves wobble and none keeps rising

Hermes and OpenClaw with Qwen3.8-Flash

Huang et al. · EmbodiedCLBench · arXiv 2610.13021v1 · §5.2 · Figure 5

The chores replay a recorded map of a closed-source simulator called TongSim, and every step is a four-way choice. Each result is a single success rate with no spread, and the plateau was measured on two harnesses with one model.

The code and task data are public, and it runs against model APIs with no GPU. For agents that learn from their own runs, the open problem is the second practice chore, which adds no consistent gain.