GPT-6 Sol trains virtual Microduck in hours, not weeks

The Microduck experiment is a first-person test of a broader shift: OpenAI's latest models are now good enough at understanding 3D representations of the physical world to translate them into useful code, agentic actions, and robot training. Knight notes that he previously wrote about using agents and coding models to configure, program, and debug physical robots, but that OpenAI's newest model went beyond writing code — it used Nvidia's Isaac Sim to feed instructions, watch videos of the virtual robot in action, and suggest tweaks to its training regime.

That same capability is showing up in academic labs. When OpenAI released Astra this past September, UC Berkeley professor Ken Goldberg and his students wondered if it could help with the detailed simulations required to train robots. One lab member gave Astra access to an Nvidia tool called Newton, showed it a video clip of a real robot in action, and asked if it could create a simulation of the scene. Goldberg says that within an hour it had produced a full 3D simulation with all the parameters set — the position of the camera, the lighting, the background, all the materials.

Nvidia researcher Jim Fan, who has collaborated with Goldberg, says the way AI can now control robots mirrors how software agents function, with LLMs using visual language action (VLA) models much as they access different software on a computer. This week Fan, Goldberg, and others open sourced an agentic framework for robots called Aspire, which allows a robot to discover and use new skills; in one example, GPT-6 Astra figured out how to use a pair of robot arms to solve a Rubik's cube. Knight cautions that getting robots to work reliably and efficiently is still extremely difficult and requires significant engineering expertise — the vision system on his robot duck does not work well enough to be deployed in a real product — but he suggests the ease and speed of training robots with AI could soon have an enormous impact on manufacturing and e-commerce, and Goldberg says a number of industrial robotics and automation firms have shown an interest in the approach.