TL;DR
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
Researchers report that GPT-6 Astra edited two weight matrices in the open model Qwen3.5-0.8B, raising its MATH-500 score from 7.9% to 48.1%. The change reduced answers that hit the benchmark’s 32,768-token limit from 93.0% to 33.1%; the work is an early blog-post result, not a peer-reviewed study, and the loop problem remained in some answers.
Researchers report that GPT-6 Astra edited the weights of the open model Qwen3.5-0.8B to reduce repetitive reasoning on math problems, lifting its score on MATH-500 from 7.9% to 48.1%. The change used no gradient descent, but it did not eliminate the model’s looping: 33.1% of its answers still reached the benchmark’s 32,768-token limit, according to a November 2 research blog post.
The post, titled “Sparks of Weight-Space Programming,” was written by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, who are listed with MIT CSAIL. The authors call the results “very early”. The post is not peer-reviewed, and it is not on arXiv. OpenAI is not listed as an author, and the post reports no statement from the company about the experiment.
According to the researchers’ logs, Astra identified a direction associated with repeated output in the model’s multilayer perceptrons at layers 15 and 19. It subtracted that direction from 64 output columns in each layer, changing 131,072 weights, or about 0.016% of the model. The edited Hugging Face version changes only the two corresponding down-projection weight tensors, according to the source material.
The benchmark figures show both the improvement and its limit. Before editing, 93.0% of MATH-500 answers ran to the token cap; afterward, 33.1% did. The researchers report a score increase from 7.9% to 48.1%. They also say that “fixing loops” is not the same as making a model better at math: the result suggests that reducing a failure mode helped the model show more of its existing capability, rather than establishing a new mathematical ability.
GPT-6 Astra Edited Qwen3.5-0.8BA targeted fix for looping
Researchers report that a small weight edit reduced repetitive answers and raised the model’s MATH-500 score. The early, unreviewed result leaves loops and open questions about reliability.
What changed
The researchers describe a targeted edit to two layers of the open model.
Astra identified a direction linked to repeated output in layers 15 and 19.
It changed 131,072 weights without gradient descent or fitting to a loss.
The reported MATH-500 score rose, while some answers still hit the token cap.
A measurable but incomplete fix
The benchmark shows fewer runaway answers, not a complete end to looping.
- 93.0% of answers reached the 32,768-token cap.
- MATH-500 score was 7.9%.
- 33.1% still reached the token cap.
- MATH-500 score was 48.1%.
What teams should take from it
Accessible model weights may allow targeted edits to address specific failures.
A focused intervention
The change touched a small share of weights and improved results on this benchmark.
Broader effects are unclear
The report does not establish transfer to other tasks or effects on other behaviors.
Early research only
The authors call the results very early; the post is not peer-reviewed.
A Narrow Fix With Measurable Gains
The experiment matters because it suggests that an AI system may be able to inspect and modify an open model’s weights using a targeted edit, rather than conventional training with gradient descent. For teams running models with accessible weights, a small change could address a specific failure while leaving most parameters untouched. The reported gain is substantial on this benchmark, though the remaining 33.1% token-limit rate shows the patch was incomplete.
The method also makes careful evaluation important. The improvement was measured on MATH-500, and the authors say they withheld the final benchmark problems from Astra. They gave the system separate practice problems and repeated feedback through forward passes. That setup is more involved than a single prompt or an automatic one-step fix, and the results do not establish how well the edit transfers to other tasks or real-world use.
Weight edits can also change model behavior in ways that matter beyond accuracy. The post cites prior research on removing refusal behavior by deleting a direction in open chat models. That example shows why targeted editing could have both useful and consequential applications; it does not show that this math-loop edit caused such changes.
Products Worth Considering
LG UltraGear 27GX704A-B 27-inch QHD OLED Gaming Monitor
Samsung Galaxy Tab A11+ 11-Inch Tablet, 6GB RAM, 128GB Storage, Expandable, Gray
Bose TV Speaker Compact Soundbar with Bluetooth and HDMI-ARC, Black
As an affiliate, we earn on qualifying purchases.
How the Weight Search Worked
The researchers say Astra could inspect the 24-layer model’s weights through a Python harness running on a CPU, but could not run the model freely. Each forward pass let it observe answers to as many as 512 problems or read up to 1,024 texts. The harness allowed up to 300 code runs, and the model had practice material from GSM8K and MATH that was separate from the final test sets.
The team tested budgets of 10, 25 and 50 forward passes. The reported winning run used 50 passes and involved 158 turns, 80 Python runs and 20.3 hours, according to the logs described in the source. With 10 and 25 passes, the reported MATH-500 scores were 12.3% and 14.1%. The post says the researchers barred gradient descent, backward passes and fitting weights to a loss.
The base model’s repetition was apparent early: 85 of its first 96 practice answers reached the 32,768-token cap, the authors report. Qwen’s model card also warns that the model is prone to thinking loops. The post covers seven other experiments, including cases where Astra did not identify which of two fine-tuned models had been trained to flatter a user.
““Fixing loops, however, is not the same as making a model better at math.””
— The authors of “Sparks of Weight-Space Programming”
Open Questions on Transfer and Reliability
The blog post reports benchmark results, but the source material provides no peer-reviewed evaluation or independent replication. It is not yet clear whether the edit improves performance on other math benchmarks, generalizes to different model versions, or holds up across repeated runs. The available figures also do not establish how the change affects other behaviors.
The remaining loop rate is material: about one in three MATH-500 answers still hit the token limit after editing. The source gives no evidence that the model’s looping was fully removed. Nor does the experiment show that a smaller search budget would reliably find a similar edit; the reported 10- and 25-pass runs produced much smaller gains.
Replication Will Test the Method
The next useful evidence would be independent attempts to reproduce the edit and evaluations on additional benchmarks, including tests of whether the gains persist without increasing unwanted behavior elsewhere. The source material does not give a date for a paper, external replication or follow-up release.
For now, the edited model and the researchers’ logs offer material that other teams can examine. Whether this approach can serve as a dependable way to repair open models will depend on those checks and on clearer evidence about its costs, limits and effects beyond this particular math task.
Key Questions
What did GPT-6 Astra change?
According to the researchers, Astra removed a direction associated with looping from weight matrices in layers 15 and 19 of Qwen3.5-0.8B. The edit changed 131,072 weights, about 0.016% of the model.
Did the edit eliminate the math loops?
No. The reported share of MATH-500 answers that reached the 32,768-token cap fell from 93.0% to 33.1%, so a substantial share still hit the limit.
How much did the benchmark score change?
The authors report that the MATH-500 score rose from 7.9% to 48.1% after the edit. Those figures come from the researchers’ blog post and logs described there.
Was this model trained?
The authors say they did not use gradient descent, backward passes or fitting to a loss. Astra did, however, conduct an iterative search with practice problems, Python runs and forward-pass feedback.
Has the result been peer-reviewed?
No peer-reviewed publication is identified in the source material. The November 2, 2026 post is described by its authors as a set of very early results.
Source: Thorsten Meyer AI
