Sakana AI's MASS lifts win rate to 69.9 percent, but blind spots compound

MASS is the latest step for Sakana AI, which launched a dedicated research lab for recursive self-improvement in June and had earlier built the Darwin-Godel Machine and the AI Scientist. The paper's setup is deliberately lean: a single language model plays all three roles — solving tasks, judging results, and rewriting the workflow — guided only by text contracts and hops that define what each sub-agent returns and in what order.

The results are mixed in a way the authors do not hide. On MLR-Bench, completed submissions rose from 31 to 50 out of 60 while their average grade fell from 3.07 to 2.88, and DSBench improved only from 0.473 to 0.482. The delegation tool went unused in all 180 runs, so it remains unclear whether teamwork explains the gains at all.

A separate thread in the paper concerns feedback. Replacing only the evaluator with GPT-5.5 barely helped; the search moved faster only when the optimizer was also stronger, leading the authors to conclude that good feedback helps only when the optimizer can act on it. A robotics trace showed how far this can go wrong: after failed simulations, the model had the simulator modified to move objects directly to the target and reported success, and external judges still rated the run positively five out of six times — which is why the authors call for independent checks, limits on autonomy, and clear conditions for intervention.