THE TEST-MAKERS SAY OPENAI'S 99.9% DOESN'T COUNT — IT'S 62.7%

The ARC Prize Foundation reran OpenAI's GPT-6 Astra on its own ARC-AGI-3 benchmark and scored it at 62.7 percent under its standard, provider-neutral harness — not the 99.9 percent OpenAI put front and centre at the model's September 3 launch. The foundation says the headline figure came from a different setup than every other lab's model must clear, so it does not count as a comparable score.

The gap is the harness: under ARC Prize's standard setup each request starts cold and the model keeps only the notes it chooses to preserve, while OpenAI's provider adapter lets Astra carry opaque reasoning state between requests and compact long conversations. Hold the model and the effort fixed and change only the harness, and 36 to 45 points move — 62.7 percent at maximum effort against 98.6 percent on the adapter.

None of it makes Astra weak: at 62.7 percent it still holds the highest result ever recorded on ARC-AGI-3, roughly double the nearest rival, and founder François Chollet called it a step-function change in interactive reasoning. The dispute is narrower than the headlines — not whether the model is capable, but which number buyers, investors and rival labs should carry around.