Sakana AI's SAIL lifts robot task success to 73% via simulation rehearsal

The gap SAIL targets is between a plan that looks right and a motion that actually moves an object. A VLM can emit a sequence of gripper positions, orientations, and open/close states, and an inverse kinematics controller can reach each waypoint, yet still fail to grasp — because reaching a point is not the same as being able to grip there. SAIL closes that gap by running the candidate trajectory in a simulator, extracting frames from the resulting video, and having an evaluation VLM estimate how far each sub-task has progressed on a 0–100% scale. Those progress scores are tied back to waypoints, so the generation VLM receives a trajectory annotated with where progress stalled, not just a pass/fail label — letting it preserve high-scoring segments and revise low-scoring ones.

The search side is framed as Monte Carlo tree search where each node is an entire trajectory and each branch is a revision of it. In the same-15-candidate comparison reported by the authors, combining exploration with correction reached 65% average success across six tasks, against 51% for breadth-first generation without feedback and 37% for depth-first repeated correction. On the laptop-closing task at that budget, however, the breadth-first method scored higher — so the approach is not uniformly best.

For real deployment, the burden shifts to reconstruction: the body describes detecting objects with GroundingDINO, segmenting with SAM2, building a 3D point cloud from depth, and placing them in the simulator via camera calibration and ArUco markers. Because real execution is open-loop and was verified on one task with six trials per method, whether the loop pays off likely hinges on how faithfully a given bench can be mirrored in simulation and how often the task repeats.