Sakana AI’s Self-Improving Model Risks Reinforcing Errors in New Study
A new self-supervision method boosts performance but may bake shared blind spots into future training cycles.
Key points
- Sakana AI researchers introduced MASS, allowing one model to act as solver, judge, and optimizer.
- The method achieved a nearly 70 percent win rate against the base model after two rounds.
- Shared blind spots can cause the model to mistake errors for progress during training.
Researchers from Sakana AI and UC Berkeley have published findings on a new recursive self-improvement method called Multi-Agent Self-Supervision. The technique allows a single language model to improve its own capabilities without relying on external help or a stronger teacher model. The study highlights significant performance gains alongside inherent risks in the evaluation process.
According to The Decoder, the MASS method enables the model to solve tasks, judge results, and refine workflows independently. After two rounds of training, the improved model achieved a win rate of nearly 70 percent against the original base model. However, the internal evaluator sometimes accepts flawed steps as successes, potentially baking errors into the next iteration.
How the Method Works
The researchers designed MASS to handle the difficulty of verifying complex results, which can require costly evidence or lengthy experiments. A single plasma study may take more than 120 million CPU hours, making external verification expensive. Consequently, the model itself serves as the best available judge, despite its potential to miss its own mistakes when reviewing its work.
Inspired by Marvin Minsky’s "Society of Mind," the approach divides work among sub-agents within a single model. A workflow defines each sub-agent’s role and instructions via text appended to the prompt. A contract specifies required information and checks, while a hop sets the order of calls. An orchestrator distributes the work among these sub-agents to ensure proper execution.
Risks of Shared Blind Spots
The primary risk identified in the study involves shared blind spots across the model’s different roles. When the model misses the same errors while solving, judging, and optimizing, it can mistake those errors for progress. This reinforcement mechanism feeds incorrect patterns into the next training cycle, potentially degrading the reliability of the model over time rather than improving it.
During the inner loop, the model’s weights remain frozen while the team runs a candidate workflow. The evaluator compares the result against the current champion, and the optimizer uses that feedback to propose the next workflow. This process mainly changes contracts and hops, but if the evaluator is flawed, the optimizer will refine incorrect strategies.
What should you do?
Security and technology professionals should monitor developments in recursive self-improvement techniques for potential vulnerabilities in automated evaluation systems. Organizations deploying AI models should understand the limitations of self-supervised methods and the risks of error reinforcement. Independent verification remains critical when using models that evaluate their own outputs.
What to do and how to stay safe: Sakana AI
- Review documentation for any AI systems using self-supervision to understand how they handle error detection and correction.
- Implement independent verification steps for critical outputs generated by self-improving models to catch shared blind spots.
- Monitor vendor updates for patches or improvements to evaluation mechanisms in recursive self-improvement frameworks.
- Assess the risk of error reinforcement in workflows where a single model handles solving, judging, and optimizing tasks.
Step-by-step guide: Prompt Injection Warning Signs: How to Spot AI Manipulation
General security guidance from the Threat Times newsroom. It is not confirmed advice from the organisations named in this story.
Frequently asked questions
What is the MASS method?
MASS stands for Multi-Agent Self-Supervision, a method where a single language model handles task solving, optimization, and evaluation without external help.
How effective is the MASS method?
The study found that the model achieved a win rate of nearly 70 percent against the base model after two rounds of training.
What is the main risk of this approach?
The main risk is that shared blind spots can cause the model to mistake errors for progress, reinforcing mistakes during training.
