Agent-Reflex
Agent-Reflex trains an 8B agent to recover when its MCP tools fail, and it gets derailed half as often
Guanting Dong, Xinyu Yang, Jiajie Jin and colleagues, with Zhicheng Dou · Renmin University of China, National University of Singapore
Agent-Reflex teaches a small open model to notice when an MCP tool fails or a user corrects it, and to recover. It comes from Renmin University of China and the National University of Singapore.
Tool calls fail often
About one tool result in seven came back as an error
24,510 errors in 177,700 tool returns · agents replayed on 9 public benchmarks
Real tools fail all the time. Replaying agents on nine public benchmarks, about one tool result in seven was an error. A third were services that were down, followed by permission failures and timeouts.
Frontier models drop
GPT-5.2 and Claude 4.5 Sonnet finish fewer tasks when tool results are faulty
Success rate, %, clean → tool noise · AgentNoiseBench on τ²-Bench airline, retail, telecom
Frontier models stumble too. On the airline, retail and telecom tasks of τ²-Bench, GPT-5.2 drops from about 80 to about 60 percent success once tool results turn faulty. Claude 4.5 Sonnet loses about 16 points.
Failures become rules
557 simulated MCP services, and 1,170 recovery rules mined from their failures
Failures injected into 1 tool result in 5 · 9 models from 4B to 284B attempt each task
The recipe builds 557 simulated services from popular MCP servers and injects real error types into one tool result in five. Nine models attempt each task, and the difference between runs that recovered and runs that failed becomes a written rule.
One recovery rule
One rule from the Airtable environment
Here is one rule, for a service modeled on Airtable. Failing agents acted on a search result that only looked right. The rule says to confirm every identifier, write only on an exact match, then read the record back.
Train on its own recoveries
Qwen3-8B practices recovering, then ships without the rules
Qwen3-8B fine-tunes on about 54,000 runs where it fixed its own mistakes, with the rule text stripped out and some user corrections wrong on purpose. Reinforcement learning follows, with a quarter of runs hitting failures. The finished model carries no rule library.
Derailed half as often
Noise derails Agent-Reflex half as often as its base model
Share of runs thrown off course · AgentNoiseBench, tool and user noise · injected differently from training
AgentNoiseBench injects its faults differently from the training setup. There, noise throws the base Qwen model off course in more than half its runs. Agent-Reflex goes off course in just over a quarter.
Results that look valid
The largest tool-side lead comes on results that look valid but mislead
Ahead of Qwen3-8B on all 13 noise types · 165 tasks per type
The biggest tool-side gain is on results that pass every format check but point the wrong way or leave data out. The agent only catches those by reading the content. It beats the base model on all thirteen kinds of noise.
Five tool-use benchmarks
On five tool-use benchmarks, Qwen3-8B rises from 40 to 59
Mean of VitaBench, τ²-Bench, BFCL V4, ACEBench, ToolSandbox · τ²-Bench alone: 26.2 → 62.1
The recipe also helps when tools behave. Averaged over five tool-use benchmarks, Qwen3-8B goes from about 40 to almost 60, ahead of the other 8B training recipes. On τ²-Bench its score more than doubles.
Where it stops
Still an 8B model, and still behind GPT-5.2 and Claude
It is still an 8B model. GPT-5.2 and Claude 4.5 Sonnet score higher on every test here. The rules come from one fixed snapshot of services, so changed tools need new mining, and the recipe has only been tried at 8B.
Model, data and code
The Agent-Reflex model, training data and code are public
Training: 16 × 80 GB GPUs for fine-tuning, 8 for RL
The checkpoint, both training sets, the rule library and the code are public under the MIT license. Retraining it takes sixteen 80-gigabyte GPUs for fine-tuning and eight for reinforcement learning.
