LEGO-Anything's LEGO-Bench: GPT-6 Astra hits 53.4%

The work is built around a simple shift: instead of generating a 3D scene directly, LEGO-Anything has a coding agent produce a Blender program. Because the output is code, objects, geometry, layout, and camera position are captured explicitly, and the scene can be run, checked, and modified. To measure how well this actually works, the researchers built LEGO-Bench from professionally built simulator scenes, giving the inputs a natural look while keeping exact geometry, depth, and object assignments as an automated answer key.

The benchmark exposed a sharp split. All six tested GPT configurations delivered a working scene almost every time, but accuracy varied widely: GPT-6 Astra, the best tested model, hit 53.4 percent on indoor scenes and 39.6 percent on outdoor scenes, while weaker configurations scored around 15 percent. More complexity meant lower accuracy, and outdoor scenes proved harder than interiors. When the researchers increased the models' reasoning budget, the GPT-6 variants improved a lot — Astra jumped from 32.3 to 61.8 percent on an office test subset.

Digging into why, the team found poor initial attempts, revisions that undid earlier progress, and unreliable self-assessment as the most common issues. That last point is the most telling: when models had to pick which of two versions better matched the original, their judgments landed near or below chance, even for their own scenes. The plugin they built in response improved all six models, with weaker agents gaining up to 62.7 percent while the top model gained only about two percentage points. The open question is whether these scenes can serve as a basis for standard vision tasks; the authors report usable but unremarkable results, with a big gap remaining between a working result and a faithful reconstruction.