GPT-6 Astra changed two matrices inside Qwen3.5-0.8B and, in the authors’ evaluation, its MATH-500 score rose from 7.9% to 48.1%. GSM8K went from 13.8% to 67.1%, and strict instruction-following on IFEval rose from 21.0% to 62.3%.
The result comes from a November 2, 2026 research report by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, all listed with MIT CSAIL affiliations. They call the approach weight-space programming: using an AI model to inspect, reason about and directly edit another network’s weights.
The agent touched 131,072 weight entries, about 0.016% of the model’s nominal 800 million parameters. It used Python to calculate the replacement values, then submitted the edited tensors for testing. The target model was not fine-tuned with gradients during this editing process. The released results and edit manifest make the claim unusually concrete: there are weight changes, run logs and benchmark files to inspect.
The most convincing explanation is also more specific than “the agent taught a model math.” In this setup, the original model often got trapped in long stretches of reasoning and failed to produce a final answer. The edit substantially reduced that failure. Many answers that had been cut off became correct answers. That is a valuable result in its own right: a capability hidden behind a bad failure mode can be as inaccessible to a user as a capability the model lacks.
The important shift is the agent’s job. It analyzes and edits the numerical machinery that will run future requests. A saved weight change can persist across deployment, rather than requiring the editing agent to answer every question. That possibility makes the result interesting even before claims about replacing conventional training enter the discussion.
What it means to program weights
A neural network stores what it has learned in arrays of numbers called weights. During inference, the model repeatedly applies those arrays to its current internal state. In a language model, those operations help determine which token comes next and how the model carries information forward through its layers.
A prompt influences the computation by changing the input. Fine-tuning typically changes weights through a training procedure that measures an error and uses gradients to adjust parameters. A direct weight edit changes selected values through a calculation chosen by the researcher or an agent. These are descriptions of different workflows; direct edits can still use data, feedback and optimization.
For a developer, imagine diagnosing a server that repeatedly enters a retry loop. You could write a better client request, retrain a policy to avoid the loop, or investigate the internal state and change a component implicated in the failure. Weight-space programming attempts something closer to the third approach, except the “component” is a set of numbers distributed through a neural network, and their behavior is much harder to interpret than a conventional function.
The editing agent had a persistent Python environment, the target model’s tensors, practice problems and tools to run controlled inference. Python did the array arithmetic; Astra did not print hundreds of millions of weights into a chat response. The human-designed task also advised it to diagnose failures, test causal interventions and confirm gains on held-out practice items. The initial task instructions describe the permitted tools and the protected parts of the model.
Following the repetition signal
The successful run began with a practical symptom. On many math problems, Qwen generated reasoning until it reached a 32,768-token ceiling without finishing. The authors’ completed MATH-500 evaluation records 93.0% of original responses hitting that ceiling. A ceiling hit is not, by itself, a perfect loop detector, but it is a serious obstacle when the grader needs a final answer.
The agent’s final submission describes comparing internal activity from productive and repetitive contexts, estimating directions associated with repetition and finding components that wrote strongly along them. Its final edit targeted selected columns in the output weights of two feedforward, or MLP, layers. Each selected column corresponds to one MLP neuron’s contribution to a 1,024-dimensional residual stream, the shared representation carried through the model.
The run’s final report expresses the operation as:
W_new = W_original - 6 × outer(u, mask64 × (u @ W_original))
Here, u is the estimated unit repetition direction for that layer; u @ W_original measures how strongly the columns write along it; and mask64 retains the 64 selected columns for that layer. The outer product constructs an adjustment from those pieces. The factor of six matters: this was a scaled intervention, not merely the removal of one unscaled projection.
The experiment history records intervention and control tests before the final choice was frozen. Those tests support the authors’ interpretation within this run, although they do not prove that a single universal “loop direction” exists in language models. Kingy has inspected the released records and recalculated the figures, but has not replayed the model or the intervention experiments.
The selected matrices are at zero-based layers 15 and 19. Each has shape [1024, 3584], and 64 columns were targeted in each. The arithmetic is:
2 layers × 64 columns × 1,024 entries = 131,072 touched entries
131,072 ÷ 800,000,000 × 100 ≈ 0.016%
The denominator uses the model’s nominal 0.8 billion parameters. The released edit manifest counts 130,188 entries bitwise different after bfloat16 conversion; some calculated changes rounded back to the original. The frozen-weight manifest also reports relative changes of roughly 10.6% and 8.1% across the two edited matrices. A small count of changed entries says where the edit landed; it does not establish that its effect was small or that every other behavior was preserved.
The results behind the 6× and 3× claims
The headline multipliers come from dividing the edited score by the original score. On MATH-500, 48.1 divided by 7.9 is approximately 6.1. On IFEval, 62.3 divided by 21.0 is approximately 3.0. The absolute scores and evaluation setup matter more than either multiplier.
| Measure | Original | 50-pass edit | Evaluation size |
|---|---|---|---|
| MATH-500 accuracy | 7.9% | 48.1% | 500 questions, four samples each |
| GSM8K accuracy | 13.8% | 67.1% | 1,319 questions, two samples each |
| IFEval strict prompt accuracy | 21.0% | 62.3% | 541 prompts, two samples each |
| MATH-500 responses reaching token cap | 93.0% | 33.1% | 500 questions, four samples per version |
The aggregation code calculates mean accuracy across sampled responses, rather than pass-at-k or majority voting. The gains are 40.2 percentage points on MATH-500, 53.3 points on GSM8K and 41.3 points on IFEval, using rounded scores. The original results, edited results and evaluation code provide the underlying measurements.
The same release reports gains on AMC23, Minerva Math, OlympiadBench, a 1,000-question MMLU-Pro subset and GPQA Diamond. Its additional AIME evaluations remained weak: 0.625% on AIME 2024 and 2.292% on AIME 2025 after editing. Those results limit any sweeping description of the model as a newly strong mathematician. They also show why a relative improvement needs a baseline.
The IFEval result is particularly interesting because the agent was working on math failures. But “strict prompt accuracy” means every checked constraint in a prompt must pass; it is not the percentage of individual instructions followed. The grading protocol also matters. The evaluation checks the answer after the model closes its thinking section. If thinking never closes, the answer is empty. A model that finishes more often can therefore gain instruction-following points without acquiring every instruction-following skill anew. The IFEval paper describes the benchmark’s verifiable constraints; the project’s evaluation code specifies how the model’s answer is extracted.
The authors’ held-out practice analysis gives a more direct view of the recovery. Across 1,152 paired GSM8K and MATH practice items, the original was correct on 122 and the edited model on 696. The saved analysis reports 579 items rescued and five regressions; 576 of the rescues had originally been truncated. This is an author-reported analysis of practice items, rather than an independent replay or an item-level analysis of the final benchmark. It strongly supports the narrower explanation that finishing previously unfinished answers drove much of the improvement. Submission evidence.
No gradients still required feedback
The target edit did not use gradient-based training. That is meaningful, but it does not mean the agent worked without examples or experiments. It had math practice data, inference outputs and access to internal model information. Both the editor and Qwen were already trained models.
The successful investigation used GPT-6 Astra through Codex CLI at max reasoning effort, with 50 harness-defined forward passes, 80 Python executions and 158 agent turns across four conversations. A pass was an accounting unit for a potentially substantial batch: the instructions allowed up to 512 generated responses, or up to 1,024 texts in a teacher-forced read of internal information. A paired comparison could include up to 256 examples per model. One pass should not be pictured as a single example or a single generated token. Run state, task instructions.
The authors report 20.3 hours from the first to the last turn, including 4.7 hours waiting for an unavailable model. That elapsed time is not a compute bill. The release does not establish that this approach is cheaper than fine-tuning, a sampling change or another loop-control method. Experiment README.
The practice set included training examples from GSM8K and MATH, plus historical AIME problems. The authors’ preprocessing filtered normalized exact matches and shared 13-word windows against the math evaluation sets. That is a useful contamination precaution, not proof that semantic near-duplicates or pretraining exposure were absent. The benchmark test sets were not exposed to the editing agent. Practice builder.
The release also contains 10-pass and 25-pass investigations with different final edits. These are separate attempts under different budgets, not checkpoints in one continuous training run.
What would make the comparison stronger?
The published Qwen evaluation profile used thinking mode, temperature 0.6, top-p 0.95, top-k 20, a presence penalty of 1.5 and a 32,768-new-token limit. As of November 2, the Qwen model card warns about thinking loops and lists different recommended thinking-mode sampling, including temperature 1.0. Its reported benchmark scores use different conditions and cannot be compared directly to this experiment’s numbers.
A useful next test would run the original and edited models under matched recommended sampling, alongside practical loop controls, with the same prompts, seeds where appropriate, grading and token limits. That would help determine how much improvement the edit adds over changes a developer can make without touching weights.
The edit did not eliminate all failure. One-third of edited MATH-500 responses still reached the cap, and the final run state records whitespace-tail degeneration in 51 of the 1,152 edited practice outputs. Wrong arithmetic and missing answers remained. Aggregate gains across benchmarks also do not establish that every individual response improved or that unmeasured capabilities were unaffected. The five paired practice regressions are a reminder to look at the cases behind the averages.
A follow-up on Gemma 3 270M IT is just as useful for setting expectations. That model rarely reached its shorter token limit in the baseline, and the same editor did not establish a meaningful math improvement. Format-agnostic MATH-500 accuracy was 6.8% before editing and ranged from 7.0% to 7.7% across the tested budgets. Gemma used a different architecture and evaluation setup, so the result does not prove direct editing can only recover loop-suppressed ability. It does show that the Qwen outcome did not automatically carry over. Gemma follow-up.
The broader project and earlier work
The project also asks whether weights reveal what a network does. In a controlled test, Astra identified the training corpus of three small character models—Python, romaji or English—from their middle transformer blocks, in two attempts per model. Input and output layers were withheld, and forward passes were forbidden. That supports a limited but intriguing interpretability result, rather than unrestricted access to a model’s knowledge. Language-identification experiment.
In one controlled car experiment, an agent received an environment description and a recorded failure, then edited a policy that went from zero successes in 256 trials to 1,024 successes in 1,024 trials. That is an impressive result for that specific environment, not evidence that arbitrary driving systems can be repaired this way. Car experiment.
For 9×9 Go, the 50-pass written policy won 74.2%, 70.0% and 78.5% against three seeds of a same-size PPO baseline, over 2,000 games per seed. That baseline was tuned across three learning rates and trained for 20.48 million games. The zero-pass written policy lost to those same-size seeds, despite narrowly beating two smaller PPO policies. The distinction matters when judging the benefit of feedback and the strength of the comparison. Go experiment.
The inspection experiments had limits too. In a small sycophancy test, four rounds of inspecting paired Gemma variants’ weights produced no reliable discrimination; providing attention patterns led to three correct identifications out of three, with one final confidence only 55%. Those were deliberately trained toy variants, not audits of commercial systems. Sycophancy experiment.
Direct intervention in model weights has precedents. ROME studied targeted edits to factual associations in transformer feedforward weights. Other researchers have investigated direction-based interventions, and Tracr compiled specified programs into transformer weights. The distinctive proposal here is to use a general-purpose frontier AI agent as investigator and editor. In the Qwen case, it developed a numerical intervention in the model’s computation rather than writing code that trained the target with an optimizer. Some smaller experiments allowed no feedback at all. The project does not establish recursive self-improvement or make prior model-editing research obsolete.
How to inspect and rebuild the released edit
Developers can inspect the edit without rerunning Astra. Use the public Qwen3.5-0.8B base revision 2fc06364715b967f1860aea9cf38778875588b17, retaining the model.language_model.* tensor names. The released application script needs its neighboring st_numpy.py dependency and checks original and reconstructed tensor hashes. Patch application script.
After obtaining the repository and the matching base model.safetensors, the following is the released script’s basic invocation. Kingy has not run this command:
python exp8_math/code/apply_edit.py \
/path/to/base/model.safetensors \
exp8_math/runs/p50c/edit \
/path/to/edited/model.safetensors
A .safetensors output copies the base and replaces edited tensors; a directory output writes evaluated bfloat16 values as float32 .npy files. Keep the original checkpoint for comparison and reversal.
Rebuilding tensors is only the first part of reproducing a score. A meaningful comparison also needs a compatible model configuration, tokenizer, thinking template, numerical precision, decoding settings, prompts and graders. The experiment’s release notes show why this is not a turnkey recreation: the editing harness and some grader modules are absent, evaluation scripts retain internal paths, and many intermediate activations and generated practice files are omitted. The authors also disclose that earlier prompt trials are not included, which limits assessment of selection effects. Repository disclosure.
The immediate research opportunity is a controlled comparison across more models and failure modes: preserve the released edit, measure per-item gains and regressions, and test it against simpler decoding and loop interventions under the same conditions. That would show when an agent’s access to the weights earns its extra complexity.
Chart: Kingy.ai, using author-reported results from the released evaluation files. Reporting and source inspection: November 2, 2026.
The Kingy Brief
Get the free AI price sheet.
26 AI plans and models on one page, from official pricing pages, updated every month. Free when you join The Kingy Brief: AI changes, original tests and one practical thing to try, Fridays at 09:00 Vancouver time.
Free · Double opt-in · Unsubscribe anytime
Sign up for The Kingy Brief on Beehiiv.
Signup help and newsletter schedule
Signup form provided by Beehiiv. After submitting, check your inbox for "Confirm your subscription to The Kingy Brief" and open its confirmation link. Your price sheet download is on the page that opens. Check Spam or Promotions if you cannot find it.
Fridays at 09:00 Vancouver time: source-checked AI changes, original tests and one practical thing to try. We skip a week when there is not enough verified material. Free. Unsubscribe anytime.