8-bit AI training runs fine for 100 steps, then writes garbage
“Towards Full Pipeline FP8 Reinforcement Learning for LLMs” · arXiv 2609.22870 v1 · Sep 2026 · preprint
Train an AI model with eight-bit numbers, and a hundred rounds in, it starts writing garbage. That format, FP8, stores every number in eight bits instead of sixteen, so the math runs faster and each number gets rounded more coarsely. The training is reinforcement learning. The model tries an answer, gets a grade, and gets nudged toward what scored well and away from what scored badly. The model is Qwen three from Alibaba, practising on math. Qwen three has eight billion parameters, the numbers a model adjusts as it learns. The standard sixteen-bit run, called BF16, stays steady. This paper traces the blow-up to rounding noise that switches off the penalty for garbled answers.
The standard correction is on in every run, and the answers still turn to garbage
“Towards Full Pipeline FP8 Reinforcement Learning for LLMs” · arXiv 2609.22870 v1 · Sep 2026 · preprint
In this kind of training, one copy of the model writes the answers and a second copy learns from the grades. Earlier work put only the writing copy in eight bits, and a standard correction handled the small disagreement between the two copies. That setup trains as steadily as full precision. This paper keeps the correction on in every run and puts the learning copy in eight bits too. The blow-up comes anyway.
The safety check stops pushing down a bad answer’s word once it has dropped 20%
“Towards Full Pipeline FP8 Reinforcement Learning for LLMs” · arXiv 2609.22870 v1 · Sep 2026 · preprint
The recipe here, GRPO, grades each answer against other answers to the same question. Its safety check comes from an older method called PPO. For every word in an answer, PPO compares two probabilities. The first is how likely the updated model makes that word. The second is how likely the same model made it before this round of updates. Dividing the first by the second gives the ratio. When an answer is graded worse than average, training pushes each of its words down. Once a word's ratio falls to zero point eight, a drop of twenty percent, PPO stops pushing it. Good answers get a matching limit on the way up.
Rounding noise tips a bad answer’s words past the limit, and their penalty switches off
“Towards Full Pipeline FP8 Reinforcement Learning for LLMs” · arXiv 2609.22870 v1 · Sep 2026 · preprint
In full eight-bit training, both probabilities in the ratio come out of rounded math, each a little off. Dividing one rounded number by another stacks the two errors together. Take a word from a badly graded answer, with a true ratio of, say, zero point eight two. That word is still inside the limit, so training should keep pushing it down. Rounding noise can make the ratio read zero point seven nine instead. PPO reads that as a word that has already dropped enough, and stops pushing. The word escapes its penalty. The same slip repeats across many garbled answers, so the model never unlearns them. The garbled answers pile up until the run falls apart. The paper calls this over-clipping.
The ratio multiplies the rounding error up to 2.9 times, and the lost penalties land on garbled answers
“Towards Full Pipeline FP8 Reinforcement Learning for LLMs” · arXiv 2609.22870 v1 · Sep 2026 · preprint
The three FP8 runs round differently: one scale for a whole table of numbers, one for each row, or one for each small block. Smaller blocks round more precisely. In the ratio, the rounding error grows by one point six five times with the finest rounding, and by two point nine times with the coarsest. On the row-by-row run, late in training, about seventy percent of the wrongly stopped words sit at the lower limit, the punishment side. Most of those words, often eighty to ninety percent, come from the garbled answers. Only six point six four percent of the garbled answers were graded better than average, so nearly all of them were due a penalty.
Moving the limit to 0.6 stops the blow-up and lowers the training reward
“Towards Full Pipeline FP8 Reinforcement Learning for LLMs” · arXiv 2609.22870 v1 · Sep 2026 · preprint
To test the lower limit as the cause, the authors dropped it from zero point eight to zero point six on the row-by-row run. The blow-up never came. The training reward, the grade the model earns while it learns, stayed below full precision. The authors read the lower reward as punishing too hard, so the limit has to land in the right place.
Calibrated Clipping sets the 8-bit limit to stop the same share of words full precision would
“Towards Full Pipeline FP8 Reinforcement Learning for LLMs” · arXiv 2609.22870 v1 · Sep 2026 · preprint
The paper's fix is called Calibrated Clipping. Every twenty rounds of training, the method runs the batch through a sixteen-bit copy of the model. The method counts the share of badly graded words that full precision would stop pushing at zero point eight. Then it sets the eight-bit limit to stop the same share. Noise pushes extra words past the old limit, so the new limit sits lower. With the finest rounding at eight billion parameters, the lower limit lands between about zero point five four and zero point six six. The method also sets the upper limit to give good and bad answers the same balance of pushes as full precision.
On Qwen3 8B the fix lifts the finest 8-bit run from 54.1 to 58.6, level with full precision at 57.6
“Towards Full Pipeline FP8 Reinforcement Learning for LLMs” · arXiv 2609.22870 v1 · Sep 2026 · preprint
On Qwen three at eight billion parameters, averaged over eight math tests, full precision scores fifty-seven point six. Plain eight-bit scores forty-six point one, forty-seven, and fifty-four point one, from the coarsest rounding to the finest. With Calibrated Clipping, the same three settings score fifty-five point nine, fifty-six point five, and fifty-eight point six. Each score is a single number, and the paper reports no repeats or error bars. Changing only how often the fix rechecks moves one score by almost two points, more than the finest rounding's one-point lead.
On a second recipe, the fix wins back most of the drop and stays 3 to 4.4 points short
“Towards Full Pipeline FP8 Reinforcement Learning for LLMs” · arXiv 2609.22870 v1 · Sep 2026 · preprint
A second recipe, DAPO, trains Qwen three at fourteen billion parameters. On AIME, a hard American high-school math competition, full precision solves fifty point nine percent of the problems. Plain eight-bit solves thirty-five point seven to forty-one point six percent. With the fix, eight-bit solves forty-six point five to forty-seven point nine percent. The fix wins back most of the drop and still stops three to four and a half points short.
The 1.5 times speed-up was measured with the fix’s full-precision checks left out
“Towards Full Pipeline FP8 Reinforcement Learning for LLMs” · arXiv 2609.22870 v1 · Sep 2026 · preprint
The paper's speed claim, up to one and a half times faster training, comes from a separate test of plain eight-bit training. The figure's caption says that test leaves out the full-precision checks. The introduction calls the fix's extra cost negligible, and the paper reports no full training time with the fix switched on. The finest rounding, the one that came out level with full precision, is also the slowest, at one point one to one point two times as fast.
On the two coarser roundings, the fix’s limits get stuck at the edges of their search range
“Towards Full Pipeline FP8 Reinforcement Learning for LLMs” · arXiv 2609.22870 v1 · Sep 2026 · preprint
The fix still needs a sixteen-bit copy of the model for its checks. On the two coarser roundings at eight billion parameters, the lower limit slides to zero point five, the bottom of its search range, and stays there. The upper limit climbs to two, the top of its range. From then on, the edges of the range set both limits, and the match to full precision stops steering them. Every model in the paper is a Qwen, from eight to thirty-two billion parameters, tested on math and one coding set.
Read it for the diagnosis of why 8-bit training stops punishing garbage
“Towards Full Pipeline FP8 Reinforcement Learning for LLMs” · arXiv 2609.22870 v1 · Sep 2026 · preprint
Read it if you train models with low-precision numbers. Section three gives two measurements to repeat on your own training: how often the check stops pushing, and how far the ratio drifts from full precision. Hold off on implementing it, because the paper links no code. The paper is by Fanchao Chen and seven co-authors, a preprint on arXiv, the open archive for research papers, posted before peer review.
