Trajectory2Gym data lifts a 4B Qwen3 agent past the stock 30B model on Tau2

Trajectory2Gym builds training data for AI agents that call tools. One pass of fine-tuning on it lifts a 4B Qwen3 model past the stock 30B one on Tau2, a set of multi-step customer-service tasks. It comes from Westlake University and Ant International.

Slide text

Trajectory2Gym data lifts a 4B Qwen3 agent past the stock 30B model on Tau2

Yihao Hu, Zhihao Wen and colleagues · Westlake University, Ant International

Hu et al. · Trajectory2Gym · arXiv 2610.33568v1 · 24 Oct 2026

A shopper asks for a bigger bottle and a dimmer lamp, then says, just the lamp. Both exchanges still go through, so keeping every successful call teaches the bottle swap too. Trajectory2Gym drops it.

Slide text

01 02 03 04 The problem

A tool call that runs can still be the wrong thing to learn

Hu et al. · Trajectory2Gym · arXiv 2610.33568v1 · Figure 1, schematic

In a pool of about 35,000 successful agent runs, two in five hold at most two decisions worth learning from. Only one in ten shows the agent recovering after a real error.

It writes new tasks around rare patterns, like a retry after a failed lookup, and runs them for real. It trains only on actions the agent could justify from what it saw. And every conversation gets equal weight.

Five Qwen3 models, from 1.7B to 30B, got one epoch of fine-tuning and no reinforcement learning. Every one finishes far more Tau tasks than the stock model it started from.

Take one retail order that has to ship to New York and swap two items. One baseline ignores the address it found. Another edits the items first, which locks the address. Trajectory2Gym changes the address first, then the items.

Slide text

01 02 03 04 The results

One retail order, three 8B agents, and only Trajectory2Gym finished it

Hu et al. · Trajectory2Gym · arXiv 2610.33568v1 · Figure 4, Tau1 Retail task 112

The Trajectory2Gym 8B model finishes about three in four Tau2 tasks. The best of six public agent checkpoints, a 32B reinforcement-learning model, finishes under half. Their training data and compute differ.

A long conversation gets cut into several training rows, so ordinary averaging lets it dominate. Weighting each conversation equally, on the same data, raised the 8B model's average by about 8 points.

On the Berkeley function-calling leaderboard, simple queries that expect one immediate call got worse at four of five sizes. Every task ran once, so small gains only show a direction. Tau1 Airline tasks also seeded training. And more repetition per pattern scored lower.

The code, the pool of about 35,000 runs and all five models are on GitHub and Hugging Face. The code alone does not reproduce the scores. If I were fine-tuning a small tool-use agent, I would try per-conversation weighting first, because it changes only the loss.