GPT-6 Astra beats Pokemon FireRed in 18 hours and 12 minutes

The Pokemon and Minecraft runs come from community and company tests that sit alongside the ARC-AGI-3 benchmark, which drops models into unfamiliar, abstract game environments where they have to figure out the rules through their own actions. The pattern ARC Prize described is that Astra translates unfamiliar game mechanics into compact symbolic descriptions and tracks objects, coordinates, rules, and planned actions in a shorthand it develops itself — observations become rules, and rules become plans. In the Fallout 3 run, that approach showed up as cycles of pausing, observing, and acting; at a bed that was supposed to heal the character, it let the game run briefly and tried again; at a keypad, it waited until the game confirmed the button press before moving on. Looking back, Astra boiled these episodes down to a single principle: check whether an input worked before repeating it.

That same trait cuts both ways. The idea of turning experience into reusable rules isn't new — the 2023 research project Voyager had GPT-4 propose Minecraft tasks, write JavaScript code for them, and revise it based on error messages, but Voyager never saw the game and worked through the Mineflayer interface, while Vals AI's run uses general computer use with screen, mouse, and keyboard. In Minecraft, Astra's self-written rule about unguarded chests stuck even when it no longer helped, and it stayed in the potato field for several hours. The outcome likely hinges on whether rule-writing can be paired with the ability to abandon a rule once the situation changes — a question that matters most to teams deploying agents on long, open-ended tasks.