MIT Researchers Get One AI to Rewrite Another AI's Weights Directly

A team at MIT's Computer Science and Artificial Intelligence Laboratory says it got one large language model to edit another model's weights directly, skipping the training process that normally produces those numbers.

Rikiya Takehi, a PhD student at MIT CSAIL, worked with Jayden Teoh, Alex Zhang, and Omar Khattab to test whether OpenAI's GPT-6 Astra could read and modify the weights of a much smaller model, Qwen3.5-0.8B, to improve its math performance. Astra changed 0.016 percent of the weights. MATH-500 scores rose as much as sixfold, and the model's IFEval instruction-following score roughly tripled.

No gradient descent ran at any point. Astra inspected the numerical values that define Qwen's behavior and rewrote them, using reasoning about what each part of the network does or should do rather than backpropagation to find new values.

"We find early signs that frontier LLMs are surprisingly good at natively reading, writing, and reasoning about neural network weights," Takehi wrote in a social media post describing the work. "Instead of training, models may treat raw weights like readable, editable code: diagnosing failures and patching them directly."

The team calls the method Weight-Space Programming. In a blog post, the researchers wrote: "All of this is possible without allowing Astra to write training loops. It uses its reasoning to figure out what each part of a neural network computes, or should compute, and changes the weights accordingly."

The paper sits alongside other recent attempts to make model internals legible. Anthropic safety fellow Joshua Fonseca Rivera has separately shown that specific neurons inside a network can be tied to a specific persona, identifiable without retraining. The CSAIL work extends that legibility claim: if a model's internal computation can be read, it may also be editable by another model that understands what it is reading.

That distinction, editing weights directly instead of nudging them through loss functions, is what the authors say could sidestep catastrophic forgetting, the tendency of a model to lose old capabilities when it absorbs new information through further training. Weight-Space Programming, as demonstrated so far, touches a small fraction of parameters, which the team argues leaves the rest of the network's existing knowledge undisturbed.

The researchers also argue this opens a path toward recursive self-improvement, models optimizing other models' weights the way symbolic optimizers like Adam or Muon currently do, except with reasoning in place of gradients. They write: "Throughout the history of deep learning, we encoded our simplistic inductive biases into tiny symbolic optimizers like Adam or Muon. This is changing: our neural network weights will soon be natively optimized by LLMs that deliberately reason about their weights."

None of this has been benchmarked at scale yet. The result comes from a single demonstration on one small model and one math task, run by the team that built the method, without independent replication or a standardized test suite. Takehi and his coauthors say their next step is building exactly that: a benchmark and a harness for running Weight-Space Programming across models, which they describe as a way to turn what currently looks like "black magic" into reproducible engineering.

The clearest illustration of what the technique can already do has nothing to do with repairing an existing model. The researchers gave Astra a blank instruction: write, from scratch, the weights of a policy network that plays the video game Breakout as well as possible. Astra produced a working set of weights with no training run behind them at all, just the model writing numbers it judged would make the game-playing network function.