Astra’s ARC Prize-verified 98.55% max-reasoning run—and the 99.95% high-reasoning result OpenAI now headlines as 99.9%—are astonishing. Neither is a clean measurement of Astra’s model weights alone.
ARC Prize has now supplied the missing denominator. Its verified results show that the originally circulated 98.6% figure maps to 98.55% for GPT-6 Astra at max reasoning on the ARC-AGI-3 Semi-Private set using a Provider Adapter harness. OpenAI’s official launch page instead reports 99.9%, selecting the best score at any effort; that maps to ARC Prize’s 99.95% high-reasoning run. The same model at max reasoning scored 62.71% with ARC Prize’s Standard harness. Meanwhile, NVIDIA’s 100% with Claude Opus 5 inside AVO remains a result on the 25 public demonstration environments.
That accounting makes the benchmark paradox more concrete, not less. Astra is plainly a major capability advance, and ARC Prize independently verified the runs. But a near-perfect score still measures a configured agent system: the gap between 62.71% and 98.55% shows how much the exposed capability changes when reasoning continuity and compaction change around the same named model. The shift from 98.6% to 99.9% also shows why reasoning effort and score-selection rules belong beside the headline.
Kingy verdict: OpenAI’s official 99.9% headline and the originally circulated 98.6% figure are both verified system results, but for different reasoning settings: 99.95% at high and 98.55% at max, each on ARC-AGI-3 Semi-Private with ARC Prize’s Provider Adapter. At max, the Standard harness scored 62.71%. This proves Astra can clear ARC-AGI-3’s bounded bar with extraordinary action efficiency. It does not isolate model weights from the adapter or prove AGI. NVIDIA’s 100% still proves public-set saturation by AVO plus Claude Opus 5, not private-set generalization.
Evidence update, 4 September 2026: OpenAI’s official Astra launch page now reports 99.9% on ARC-AGI-3 and says its tables use the maximum score at any reasoning effort. ARC Prize’s independent technical account and verified results page resolve that figure to 99.95% at high reasoning, while the 98.6% figure resolves to 98.55% at max. DataCurve’s official DeepSWE leaderboard still independently lists Astra at 74% ±3 in the shared mini-swe-agent setup. OpenAI’s GPT-6 Astra System Card confirms the release and deployment safeguards but does not itself mention ARC-AGI-3, DeepSWE or GDPval. Our broader Astra evidence tracker covers the model’s release trail and remaining product questions.
The system card does add relevant platform context. OpenAI says its Astra misalignment monitor operates across Codex, ChatGPT and the Responses API, reviewing the model’s chain of thought alongside its actions and conversation inputs and outputs. It also distinguishes stateless Responses API requests from persistent conversations: without persistent reasoning or a conversation identifier, the monitor cannot reconstruct a complete trajectory. That is a deployment-safeguard disclosure, not a benchmark-method disclosure, but it reinforces the central point that an Astra result can depend on more than model weights alone.
The benchmark changed before Astra arrived
OpenAI supplied the best reason to resist model-only interpretations in July. Using GPT-5.6 Sol on ARC-AGI-3’s 25 public environments, the company first ran the benchmark’s generic harness. That setup kept a record of moves and brief notes but discarded private reasoning after every action. It also used rolling truncation, dropping the oldest context after the interaction history crossed 175,000 characters.
The result was 13.3%.
OpenAI then used a Responses API harness that passed the previous response ID between actions, retaining reasoning across tool calls. It added compaction to compress long histories without simply deleting the oldest experience. The same named model reached 38.3% while producing roughly one-sixth as many output tokens. OpenAI’s conclusion was unusually direct: evaluations measure a model together with API settings, harness design and prompting. That experiment does not make the higher score illegitimate. It shows why the harness belongs in the product name of the result.
ARC Prize now calls Astra’s native configuration the Provider Adapter harness. It preserves opaque reasoning state between requests and uses compaction for longer conversations. At max reasoning, the adapter scored 98.55% on Semi-Private versus 62.71% for the Standard harness—a 35.84-point absolute gap with the same named model and effort. Across the 167 game-reasoning pairs solved by both harnesses, ARC Prize reports that Provider Adapter runs were about 3.66 times faster by aggregate recorded elapsed time and used 49% fewer total tokens. The public record still cannot expose Astra’s opaque reasoning or isolate every causal component, but it now quantifies the system effect directly.
NVIDIA’s 100% proves a system can saturate the public set
NVIDIA’s result is more specific. Its AVO technical account says Claude Opus 5 completed all 183 levels across all 25 public ARC-AGI-3 environments. It used 6,624 environment actions. The model received text-only 64-by-64 observations; AVO added persistent memory, a supervisor and feedback-and-recovery loops.
NVIDIA contrasts that with ARC Prize’s roughly 30% Claude Opus 5 result at High reasoning and with VISTA, another system using the same model family that required 7,542 actions. The comparison is valuable, but it is not a controlled ablation. NVIDIA explicitly notes differences in backend, observation format, memory and context. AVO’s result proves that this particular model-and-architecture combination can solve the public set. It does not isolate the number of percentage points caused by persistent memory, the supervisor or any single component.
Nor is NVIDIA’s 100% automatically stronger evidence than Astra’s 98.6% or OpenAI’s 99.9%. The percentages do not share a denominator: NVIDIA saturated the public demonstrations, while Astra’s 98.55% and 99.95% were verified on Semi-Private environments using the Provider Adapter. Astra therefore carries stronger evidence of first-contact generalization. NVIDIA’s packet remains useful evidence about what explicit persistent memory, supervision and recovery can accomplish on the public set.
Public, semi-private and private are different claims
ARC Prize is explicit about this boundary. Its ARC-AGI-3 technical report defines beating the benchmark around human-level action efficiency averaged over private environments encountered for the first time. The public environments are demonstrations: useful for learning the interface, debugging a harness and exploring agent design, but not sufficient evidence of general progress if a system has been tuned to them.
The verified ARC Prize record now makes that distinction inspectable. It lists six reasoning levels under each of two harness classes. On Semi-Private, Standard-harness scores range from 17.45% to 62.71%; Provider Adapter scores range from 96.72% to 99.95%. The originally circulated 98.6% maps to the 98.55% max-reasoning Provider Adapter run. OpenAI’s official 99.9% headline maps to the best observed result, 99.95% at high reasoning; its launch page explicitly says evaluation tables show the maximum score at any effort.
ARC Prize’s open benchmarking repository now makes another important separation. The standard harness uses provider-neutral text history and manual rolling context. Provider-adapter harnesses may use native reasoning-state continuity and compaction. The games, actions, limits and scoring can stay fixed while context handling changes. The repository says these result classes should be labeled separately: the standard harness is better for controlled provider comparisons; adapters are better for measuring a provider’s native agent stack.
ARC Prize has now adopted exactly that taxonomy. Astra’s Provider Adapter score should not be dismissed because the harness is capable, and it should not be presented as a weights-only result. The correct label is GPT-6 Astra plus Provider Adapter on Semi-Private, with the Standard-harness score beside it.
The reported 98.6% run now has the missing labels
System. GPT-6 Astra ran at max reasoning through ARC Prize’s Provider Adapter, which preserves opaque reasoning between requests and uses compaction. The matched Standard harness lets the model carry forward visible notes it chooses to keep but does not preserve provider-native reasoning state.
Split. The 98.55% result is on ARC-AGI-3 Semi-Private, not the 25 public demonstrations. ARC Prize’s verified record separately publishes public-demo and Semi-Private outcomes.
Budget. ARC Prize reports $17,332 for the max-reasoning Provider Adapter run, versus $26,098 for Standard at max; the highest score, 99.95% at high reasoning, cost $18,817. It also reports that Astra max used fewer actions than the median tested human on 96% of completed levels and 51.7% fewer actions per level on average.
Reproduction. ARC Prize has validated the entry, published per-environment results and linked replays through its verified record. Fully open reproduction remains constrained by a closed model and a Semi-Private set, but the result is no longer only a vendor launch claim.
DeepSWE confirms frontier performance, not a clear lead
DataCurve now lists GPT-6 Astra at 74% ±3 with xhigh reasoning on DeepSWE v1.1. The displayed score is consistent, after rounding, with the 74.1% shown on OpenAI’s launch page. It ties Gemini 3.8 Flash High at 74% ±1 and Claude Opus 5 Max at 74% ±4 by rounded point estimate, while GPT-5.6 Sol Max sits at 73% ±3. The uncertainty ranges overlap, so Astra has no statistically clear accuracy lead over that frontier group.
DeepSWE is designed to make comparisons cleaner. Its benchmark paper describes repository-level software tasks with executable verifiers, and every leaderboard model runs through mini-swe-agent with the same Bash tool and shared prompt. DataCurve’s 3 September changelog says it added Astra results across low, medium, high, xhigh and max reasoning at the expected launch rate card.
The efficiency figures are more distinctive than the pass-rate gap. In our 4 September update, Astra’s best row averaged $6.52, 30,000 output tokens and 29 agent steps per trial. The dashboard marked updated 22 September lists $4.43 for the xhigh row, still at 30,000 tokens and 29 steps. These dated observations do not establish why the displayed cost changed. Gemini 3.8 Flash averages $2.36, 143,000 tokens and 166 steps; Claude Opus 5 averages $11.84, 118,000 tokens and 99 steps; GPT-5.6 Sol averages $6.46, 60,000 tokens and 61 steps. The shared setup makes DeepSWE a cleaner model comparison than ARC-AGI-3. Astra is a top-tier coding model with unusually low token and step counts, not an accuracy category of its own.
Why the missing GDPval result matters—and why it would not settle AGI
OpenAI built GDPval to evaluate economically valuable work products across 44 occupations and nine industries. A strong result would be relevant evidence for a claim about broadly useful intelligence, rather than competence inside games, repositories and specialist technical tasks.
But GDPval would not be a magic counterweight. OpenAI describes the initial benchmark as a one-shot evaluation; it does not capture the iterative, long-horizon collaboration where Astra is supposed to excel. The fair conclusion is narrower: neither OpenAI’s official launch page nor the system card publishes a GDPval result, so the public launch packet does not quantify Astra’s breadth on OpenAI’s own economic-work benchmark. It does not mean Astra performed poorly or was never tested.
What each result actually proves
| Result | What the public evidence supports | What remains unproven |
|---|---|---|
| Astra: 98.6% reported / 99.9% official | ARC Prize verifies 98.55% at max reasoning on ARC-AGI-3 Semi-Private with Provider Adapter for $17,332; Standard at max scored 62.71% for $26,098. OpenAI’s official headline selects the 99.95% high-reasoning result. | The weights-only contribution, provider-internal reasoning state and fully open reproduction on the non-public set. |
| AVO + Claude Opus 5: 100% | NVIDIA documents complete coverage of 183 levels in the 25 public environments using 6,624 actions. | Private-set generalization and a controlled estimate of how much AVO—not other configuration differences—caused the gain. |
| GPT-5.6 Sol: 13.3% → 38.3% | OpenAI demonstrates that retained reasoning and compaction can nearly triple a public-set score while reducing output tokens. | Whether the same magnitude of harness effect applies to Astra or unseen private tasks. |
| Astra: 74.1% DeepSWE | DataCurve lists GPT-6 Astra xhigh at 74% ±3 in the common mini-swe-agent setup. Our 4 September update recorded $6.52, 30,000 output tokens and 29 steps per trial; the dashboard marked updated 22 September lists $4.43, as discussed above. | A statistically clear accuracy lead over Gemini 3.8 Flash or Claude Opus 5; DataCurve’s public table rounds the point estimate to 74%. |
The paradox is not that agent scaffolding makes a benchmark fake. Long-horizon intelligence may require memory, state management and recovery, just as useful computing systems require more than a processor. The mistake is presenting a system score as though it cleanly identifies the capability of one component.
Astra’s result should raise expectations—and ARC Prize’s disclosure packet now meets much of the standard the launch headline initially lacked. It names the split and harness class, reports paired Standard and Provider Adapter runs, publishes costs and action-efficiency evidence, and provides verified per-environment outcomes. OpenAI’s official 99.9% headline selects the best result across reasoning efforts rather than the 98.55% max-reasoning run, an additional reporting choice readers need to see. DeepSWE separately confirms top-tier coding performance in a shared setup, with unusually low token and step counts but no clear accuracy lead over its closest peers.
The resulting conclusion is stronger and narrower than “AGI achieved.” Astra clears a difficult bounded benchmark with human-beating action efficiency, while the paired harness results show that memory and context orchestration remain part of what the score measures. The launch number now survives scrutiny precisely because the system around the model is visible.
The Kingy Brief
Get The Kingy Brief.
AI changes, original tests and one practical thing to try. Fridays at 09:00 Vancouver time.
Free · Double opt-in · Unsubscribe anytime
Sign up for The Kingy Brief on Beehiiv.
Signup help and newsletter schedule
Signup form provided by Beehiiv. After submitting, check your inbox for "Confirm your subscription to The Kingy Brief" and open its confirmation link. Check Spam or Promotions if you cannot find it.
Fridays at 09:00 Vancouver time: source-checked AI changes, original tests and one practical thing to try. We skip a week when there is not enough verified material. Free. Unsubscribe anytime.