FACET
Shanghai AI Laboratory · USTC · Fudan University
One paper today. FACET, from Shanghai AI Lab, U S T C, and Fudan. It answers a question every coding agent team has right now. Where does trustworthy training data for terminal agents come from, when writing tasks by hand does not scale. Here is the first pass.
The Claim
The claim, in one sentence. You can mass produce terminal agent training tasks that you can actually trust, if the instruction, the solution, and the tests are all grounded in one real, running container. Not three artifacts written separately and hoped into agreement. One environment that all three must answer to.
The Method
The method is a three stage pipeline, and this is the paper's own figure. Stage one collects over seventy one thousand real skills from open agent repositories and distills them into scenarios. Stage two rebuilds each scenario into a goal, a context, and matched references for the instruction and the solution. Stage three is the idea that matters. It builds the actual Docker environment first, validates that it runs, and only then generates the instruction, the solution, and the tests against that live state. When a test fails, the pipeline repairs the artifacts and tries again, so a task that survives is executable by construction.
The Evidence
17.60 → 24.72
27.34 → 35.58
40.82 → 47.57
Does the data actually work. Fine tuning on it lifts the four billion parameter Qwen model by seven point one two points on Terminal Bench, from seventeen point six to twenty four point seven. The nine billion model gains eight point two four. And the twenty seven billion model reaches forty seven point six, up six point seven five. Here is the striking part. The training set behind all of that is twelve hundred trajectories. Not millions. The leverage comes from density, nearly twenty three executable checks on every single task.
The Catch
Now the catch, and there are three. Every training gain is on Qwen three point five, and nothing above twenty seven billion parameters. Whether frontier scale models benefit is untested. This is the terminal domain only, measured on one benchmark with one agent scaffold, and to the authors' credit they claim nothing beyond that. And the tasks are genuinely hard. The teacher agent solves only about twenty one percent of its own tasks, so harvesting training trajectories is expensive. One more thing. The paper ships without a dedicated limitations section, so this scoping is ours, not theirs.
The Verdict
The verdict. Read it. If you train or evaluate coding agents, the environment first idea is worth an hour of your weekend, and the failure analysis alone will sharpen how you write verifiers. If the six thousand task set ships publicly, this graduates to implement it.
Sources
arxiv.org/abs/2608.18580
Discovery — Hugging Face daily papers, August 21
huggingface.co/papers
The paper is FACET, from Shi and colleagues at Shanghai AI Lab. The full text is on arXiv, linked right here. That is the first pass. Back Monday with the next paper.
