World Labs turns one robot task into thousands of simulated variants for training

World Labs has developed a system called R2S2R that addresses a fundamental bottleneck in robot deployment: the need for vast amounts of diverse training data. Rather than requiring robots to learn from expensive, hard-to-control real-world experiences, the system captures a single real-world task—including the robots, sensors, environment, and task demonstrations—and reconstructs it as an interactive virtual world that behaves physically like the original.

The engine then generates thousands of controlled variations from that single task by systematically changing lighting conditions, object positions and counts, surrounding environment, physical properties like friction, and camera angles. To verify accuracy, World Labs runs the same action sequence in both simulation and reality side by side, comparing observations, object movements, and outcomes. The test tasks included rigid, movable, and deformable objects: cable routing, inserting an elastic cable end into a hole, and packing a box with both hands.

World Labs tested control models trained entirely in simulation on multiple robot platforms, including ALOHA, an open-source dual-arm design from Stanford that costs a fraction of commercial systems and has publicly available blueprints. According to the company, the models each ran for one hour across four additional robot platforms without any human intervention, performing tasks such as wrapping a power cord around a refrigerator with both hands and precisely repositioning test tubes or separating thin objects like markers and pencils from a dense jumble.

A critical validation came from testing whether simulation rankings of models match real-world performance. World Labs evaluated a two-handed cube handoff task using 2,000 simulated runs and 100 real runs per checkpoint across different model types, including GR00T N1.6 and π₀.₅, and across different training stages. The simulation reproduced both borderline cases where the robot barely grasped the cube by its edge and matching failed attempts. Across known and previously unseen cube positions, model rankings in simulation and reality remained largely consistent. This consistency suggests development teams can filter out weak model versions in simulation, reserving expensive hardware tests for the most promising candidates.

World Labs was founded in 2024 by AI researcher Fei-Fei Li with a mission to build models with spatial intelligence that understand the three-dimensional physical world. An early system generated walkable 3D environments from single photos. The company raised one billion dollars in venture capital to extend its world models into robotics and science, with the R2S2R engine now serving as the first concrete application of that vision in robotics. The company notes that its system is not tied to a specific control model or robot type, so a reconstructed world can be reused for new models and different robots. However, how well these results transfer to more complex environments, other robot types, and less controlled everyday situations remains an open question.