OpenAI's GPT-6 Astra Used Fewer Moves Than Human Testers on a Game-Based AI Test
The ARC Prize Foundation says OpenAI's GPT-6 Astra scored 62.7% on its ARC-AGI-3 semi-private evaluation set using the foundation's standard test harness, and 99.9% using a second setup that gives the model access to OpenAI's own context-management features. The foundation published the results Sept. 3 and ran the testing itself.
The two figures are not interchangeable. With the standard harness, which lets a model carry forward only the notes it chooses to keep, Astra at maximum reasoning effort scored 62.7% at a compute cost of $26,098, according to ARC Prize's published table. With what the foundation calls the Provider Adapter harness, which preserves the model's hidden reasoning state between requests and compacts long conversations so it can reuse earlier work, Astra at high reasoning effort scored 99.9% for $18,817. ARC Prize describes both as state-of-the-art scores.
ARC-AGI-3 is the third generation of the foundation's benchmark series. It puts an agent into unfamiliar, turn-based game environments with no instructions, and requires it to explore, infer the goal, and plan its moves. ARC Prize says human participants solve 100% of the environments.
The foundation also counted how many moves the model needed. Under the Provider Adapter harness, Astra at maximum reasoning effort used fewer actions than the human baseline on 96.0% of levels, and 51.7% fewer actions per level on average, ARC Prize reports. That baseline is the median action count among about 500 members of the public the foundation tested before launching the benchmark, who it says were not selected for puzzle-solving experience or ability.
ARC Prize says it is not claiming that Astra is AGI. It says saturating the benchmark was never meant to be proof of that, and that ARC-AGI-3's environments are deterministic and closed-ended rather than open-ended like the real world.
Across the 167 game-reasoning pairs both harnesses solved, the Provider Adapter runs were about 3.66 times faster by recorded elapsed time and used 49% fewer tokens, the foundation reports. It says both harnesses will be reported on the ARC-AGI leaderboard going forward, with each condition labeled.
