MIT team reports direct Qwen weight edit that cut reasoning loops

MIT CSAIL researchers report that a general-purpose language model directly edited Qwen3.5-0.8B’s parameters without gradient training, reducing a reasoning-loop failure that often kept the smaller model from finishing answers.

In the team’s early weight-space-programming experiments, a 50-forward-pass edit raised reported MATH-500 accuracy from 7.9% to 48.1% and reduced the share of responses that reached the 32,768-token limit from 93.0% to 33.1%. The researchers characterized the gain as loop repair: the edit helped Qwen complete work it could already perform rather than giving it new mathematical knowledge.

The team calls the method weight-space programming. Instead of updating a model with an optimizer and gradient descent, one language model gets tools to inspect and change another network’s numerical parameters in response to a natural-language objective. For this experiment, GPT-6 Astra worked in a CPU-only Python environment with Qwen’s weights and practice problems from GSM8K and MATH. The researchers barred backward passes, autograd, optimizers and iterative loss fitting, and Astra did not see the evaluation benchmarks while editing the model.

In the strongest run, Astra traced the looping behavior to a direction written by multilayer perceptrons in layers 15 and 19. It applied a directional edit to 64 output columns in each layer, targeting 131,072 weights. The released manifest says 130,188 values ultimately differed from the base model after bfloat16 rounding. The researchers described the targeted set as 0.016% of the roughly 0.8-billion-weight model. The same evaluation found reported GSM8K accuracy rose from 13.8% to 67.1%, while strict prompt-level IFEval accuracy increased from 21.0% to 62.3%.

The result remains specific to the team’s evaluation setup. Qwen’s model card already warns that the 0.8B model is more prone than other Qwen3.5 variants to enter loops in thinking mode and may fail to terminate properly. The research report has not been peer reviewed, and the opened evidence included no independent replication of the full experiment suite.

The small final patch came after a substantial search process. The 50-pass run took 158 conversational turns, 80 Python executions and 20.3 hours of wall-clock time, including 4.7 hours during a service outage. The sources offer no controlled comparison showing that direct weight editing is generally faster, cheaper or more effective than fine-tuning, retraining or simpler changes to generation settings.

The researchers released experiment logs, evaluation outputs and exact weight differences. In a follow-up on Gemma 3 270M, whose base looping rate was much lower, edited models scored between 7.0% and 7.7% on MATH-500, compared with 6.8% for the base model. The team therefore did not establish a math improvement there.