GPT-6 Astra fixed a small Qwen’s math by editing 1,023 of its weights

GPT-6 Astra rewrote 1,023 weights inside a small Qwen model, directly. On a math test it had never seen, the model went from solving fewer than one problem in ten to more than one in three. The team is at the Shanghai Artificial Intelligence Laboratory.

Slide text

GPT-6 Astra fixed a small Qwen’s math by editing 1,023 of its weights

8.35% → 37.80%

MATH-500, held out · Qwen3.5-0.8B

Junru Hu, Cheng Tang, Wei Li and colleagues · Shanghai Artificial Intelligence Laboratory, Tongji University, Fudan University

Hu et al. · Exploring Weight-Space Programming · arXiv 2611.09382v1 · 6 Nov 2026 · Table 2

In Weight-Space Programming, the agent opens the weights in a Python notebook and writes each edit as code. It gets 50 model runs to test them. There's no training step, and the output layer stays fixed.

The small model kept second-guessing itself, and 111 of 128 answers ran out of room. Astra changed one column in layer 23, so the model leans away from wait and maybe, and toward therefore and next.

With no further changes, that math checkpoint got better elsewhere too. It followed instructions more often, and went from almost never solving college science problems to about one in five. Scientific coding stayed at zero, and medical answers slipped.

Running the same search again found different edits. The first touched one column. The reruns changed one and two million values in the same layer, and one gained almost nothing.

The big gains all came on the small, weak model. The 9B model already solves nearly every math problem, and its edits added three points at most. Six of 16 searches kept the original weights.

On the 9B model, one math edit gained 30 points on practice questions. On the held-out test it scored slightly worse, and Astra kept the original weights. Those practice scores are each search's best, uncorrected for how many edits were tried.

It's two Qwen models and one editing agent, with a single search per task outside math. Long agent tasks are untested, and every check costs compute. The repository holds only the paper, but its appendix spells out every edit.

If a small model of mine kept second-guessing itself, I'd try this, since that's the failure it fixed. On a model that already works, the 9B results say not yet. The paper is on arXiv.