Base models and post-training on agent tasks
Given enough tries, base models solve more agent tasks than their post-trained versions
Changdae Oh, Sharon Li and colleagues · Meta Superintelligence Labs, University of Wisconsin–Madison, NYU, Stanford
Post-training makes a model a better agent on its first try. A team from Meta and the University of Wisconsin found it also shrinks the set of agent tasks the model can solve at all. The paper calls this the Sharpening Tax.
Fourteen model pairs, three benchmarks
14 open models, each tested before and after post-training
The study pairs each pre-trained base model with its released post-trained version. There are fourteen pairs from four open families. Each model ran up to 128 times per task on three tool-calling benchmarks.
Base models got a plain-text harness
Base models ran inside a light text harness
Base models have no chat or tool-call format, so the authors gave them a light harness. It lists tools as Python functions and takes calls back as JSON, with no worked examples. Post-trained models kept each benchmark's default setup.
Given 128 tries, the base model reaches more tasks
With 128 tries, the base model solves more WebShop tasks
Share of tasks solved at least once in 128 tries
Post-trained models win when they get one try. With more tries, the curves cross. On the WebShop shopping benchmark, the 31B Gemma 4 base model solved over 85 percent of tasks at least once in 128 tries, against 56 percent for its post-trained version.
Bigger models cross over sooner
The bigger the model, the sooner its base version catches up
Model size moves the crossover. On WebShop, the smallest Gemma 4 base model still hadn't caught up after 128 tries. The 31 billion parameter base model overtook its post-trained version after about three.
Post-training pushes tasks to the extremes
Post-training sorts tasks into always solved and never solved
The reason shows up task by task. The base model solves most WebShop tasks on some tries, so retrying keeps paying off. After post-training, about a quarter are always solved and 44 percent are never solved, so retrying stops helping.
The Sharpening Tax
The Sharpening Tax measures what extra tries stop buying
The authors turn this into one number, the Sharpening Tax. It is how much less a post-trained model gains from extra tries than its base model does. At 128 tries, the tax was positive in 36 of the 42 cases.
Eight tries predict the tax
Eight tries per task predict the tax at 32
The tax is cheap to estimate. Measured from eight tries on half the tasks, it ranked the 42 cases almost the same way as the tax from 32 tries on the other half.
PTGS sets temperature per prompt
PTGS sets each prompt's sampling temperature during RL training
The RL update itself is unchanged.
Their fix is a sampler for reinforcement learning called PTGS. It keeps a running estimate of how often the model solves each prompt. Hard prompts get sampled at a higher temperature and easy ones lower. The training algorithm stays the same.
PTGS kept more coverage
With PTGS, RL training kept more of the tasks it could solve
% of tasks solved, fixed temperature → PTGS
They tested it by training Qwen 2.5 7B Instruct on two grid puzzles, Sokoban and FrozenLake. With PTGS, both PPO and GRPO solved more tasks within 128 tries, and more on the first try as well.
What the setup leaves open
The comparison is observational
The paper calls its study observational. The labs haven't disclosed the exact training data behind these checkpoints. The two sides also ran in different setups, and thinking mode was off. Turning it on kept the same trend in the two models tested.
Where the result stops
The base model's lead needs many tries and a checker
The base model's lead has conditions. Its first-try accuracy is far lower, so you need many tries and a way to check which one worked. The smallest models often did better post-trained at eight tries or fewer. PTGS was tested on two puzzles with one model.
Report the tax next to accuracy
The authors ask labs to report the Sharpening Tax next to accuracy
The authors want post-training judged by how much of the base model's reach it keeps, with the Sharpening Tax reported alongside accuracy. The paper is on arXiv, and the code and per-task results are on GitHub.
