OpenAI triples GPT-5.6 Sol's ARC-AGI-3 score by enabling two existing API settings

OpenAI said enabling two settings already available in its API — retained reasoning and compaction — nearly tripled its GPT-5.6 Sol model's score on the ARC-AGI-3 benchmark, taking it from 13.3% to 38.3% while cutting output-token usage sixfold, according to a July 29 blog post.

What's new

• Baseline score on the official ARC-AGI-3 harness: 13.3%
• With retained reasoning and compaction both enabled: 38.3% — roughly a 3x improvement
• Output tokens consumed dropped 6x over the same runs
• The two settings — retained reasoning and compaction — are the same options OpenAI already uses internally in ChatGPT and Codex, not new API surface
• OpenAI notes the benchmark allows context windows up to 175,000 characters, and cites a human baseline on ARC-AGI-3 of roughly 48%, for comparison

In OpenAI's own words: "turning on two API settings we use in ChatGPT and Codex—retained reasoning and compaction—tripled scores and cut output tokens by 6x."

Context

ARC-AGI-3 is the newest entry in the ARC-AGI benchmark family, designed to test novel-task reasoning and generalization rather than pattern-matching against training data — it has become one of the harder, more closely watched holdout evals for frontier reasoning models. Retained reasoning (carrying reasoning state across turns rather than discarding it) and compaction (condensing context to control token growth) are configuration options, not architecture changes, meaning the underlying GPT-5.6 Sol model did not change between the two scores.

Why it matters

The gap between a model's benchmark score under default API settings and its score under a tuned configuration is unusually large here — a 3x swing from two toggles that already existed. That suggests many API integrators evaluating or deploying GPT-5.6 Sol on long-horizon or agentic tasks may be leaving substantial capability on the table simply by not adopting the same configuration OpenAI uses in its own first-party products. It also narrows, on this specific benchmark, the gap between GPT-5.6 Sol and the roughly 48% human baseline OpenAI cites — a distance that matters for how ARC-AGI-3 is used as a signal of progress toward more general reasoning.