GPT-6 Astra scores 99.9% or 62.7% on ARC-AGI-3, depending how you ask

OpenAI's next model, Astra, scored 99.9% on the ARC-AGI-3 reasoning benchmark. It also scored 62.7% on the same benchmark. Both numbers are real, and ARC Prize published both itself on September 3, 2026. The gap comes down to which test harness Astra ran under, not a change in the model.

Tech AI Wire has covered Astra's path to release before, including OpenAI slowing its development over cyber capability concerns and pausing training after an agent breach. This is its first public benchmark result.

Two harnesses, one model

ARC Prize runs two different test setups, and the difference between them is what memory the model is allowed to keep.

The Standard harness is provider-neutral. ARC Prize's own writeup says it "carry[s] forward notes it chooses to keep with it throughout the environment," visible notes the model writes for itself between steps. The Provider Adapter harness is OpenAI-specific. ARC Prize describes it as preserving "opaque reasoning state between requests" and using "compaction for longer conversations, allowing the model to reuse prior work."

In plain terms: the neutral test makes Astra write down what it wants to remember, in the open. OpenAI's own test lets it carry hidden reasoning forward automatically, and compress long histories instead of losing them.

Harness	Score	What it allows
Standard (neutral)	62.7%	Visible notes only, written by the model itself
Provider Adapter (OpenAI's own)	99.9%	Hidden reasoning state carried between requests, plus compaction

Superpower Daily reports the Provider Adapter run was also about 3.66 times faster by elapsed time and used 49% fewer total tokens. Whatever this hidden state is doing, it is doing it efficiently.

The honest framing, from the people who ran the test

ARC Prize is not hiding the gap. It published both numbers in the same post and let readers see the harness that produced each one.

Superpower Daily puts the real question plainly. "The key question now is how much of Astra's apparent capability belongs to the model," it writes, "and how much to the context-management system around it." FourWeekMBA frames the gap in similar terms, writing that a difference "of more than thirty points... is not evidence of bad faith. It is a demonstration that a flagship benchmark number can be harness-dependent."

Outside the harness question, ARC Prize did report a genuinely new capability. On the levels Astra completed, it beat the human baseline on 96% of them, using 51.7% fewer actions than a human needed on average, according to ARC Prize. The model builds its own compact notation for each new environment, algebraic-looking shorthand for rules and states, rather than reasoning in long verbose steps. François Chollet, ARC Prize's co-founder, called the result a "step-function change" in the model's efficiency and capability, per OfficeChai.

What this means for developers

Ask which harness produced any benchmark number before you act on it, for Astra or any other model. A 37-point gap between two legitimate tests of the same model is now a documented, real phenomenon, not a hypothetical. If a vendor publishes one number without naming the test conditions, that omission is the finding.

If you evaluate models for your own use case, build your own harness rather than trusting a published one to match your deployment. The Provider Adapter's advantage, hidden state carried across requests, is exactly the kind of infrastructure most production systems do not hand a model by default. Test under conditions that resemble what your application will actually give the model to work with.

Notice what did generalize: the 96% human-beat rate and the 51.7% action efficiency held up regardless of which harness produced the headline number. When you read a benchmark result, separate two kinds of claims. The headline score moved by 37 points depending on setup. A specific, mechanism-level claim like action efficiency is more likely to reflect something real about the model itself.

Expect this exact debate to repeat. More labs are shipping agentic models with their own state-management layers. Expect the gap between a "neutral" benchmark and a "vendor-optimized" one to become a standing feature of every major release, not a one-time asterisk on this one.