OpenAI releases GeneBench-Pro, a 129-problem benchmark for AI reasoning in computational biology

OpenAI on June 30 released GeneBench-Pro, a benchmark for evaluating AI agents on realistic multi-stage scientific analyses in genomics, quantitative biology, and translational biomedicine. The paper, authored by researchers Jeremy Li and Andrew Ho at OpenAI, significantly expands on the original GeneBench with harder problems across a wider range of domains.

What's new

GeneBench-Pro covers 129 evaluations across 10 primary domains and 21 terminal subdomains, with a genomics-centered core. Unlike benchmarks that present curated, clean datasets, each problem gives an agent a raw dataset, experimental context, and a research question. The agent must navigate multiple dependent decision points—what the authors call "inferential forks"—to identify and execute the correct analysis workflow and arrive at a defensible scientific conclusion.

Key performance results on the full 129-problem suite:

• GPT-5.6 Sol Pro: 31.5% pass rate (max reasoning)
• GPT-5.6 Sol: 28.7%
• Claude Opus 4.8 (strongest non-GPT): 16.0%
• GPT-5.5: 12.0%
• GPT-5.4: 8.9%

The benchmark was externally validated at scale: 82 of 129 problems were reviewed by domain experts, whose findings prompted data and prompt modifications. Ten problems are publicly released, 50 held-out problems were given to Artificial Analysis for independent third-party benchmarking, and the remainder are retained as an internal holdout.

Context

GeneBench-Pro builds on the original GeneBench, an earlier OpenAI evaluation for multi-stage inference in genomics. The Pro version adds 29 new problems, drops three, and redesigns 54 of the remaining problems. The release is timed alongside the GPT-5.6 Sol preview—GeneBench-Pro provides the quantitative case for Sol's biological reasoning advantage.

OpenAI has been building toward life sciences AI for several years, including GPT-Rosalind for biology-focused research. GeneBench-Pro formalizes a rigorous, reproducible way to track how that capability progresses across model generations.

Why it matters

Even the best available models score below 32% on GeneBench-Pro, which points to a fundamental gap in AI's ability to perform end-to-end scientific analysis. The benchmark captures what the authors call "research taste": the chain of judgment calls about which questions a dataset can support, when early diagnostics should change the model, and when a result is decision-ready.

Models consistently identify local diagnostic signals but fail to propagate them to the corresponding analysis decision—selecting wrong estimators or persisting on initially plausible but incorrect analysis paths. Until models can reliably navigate these multi-stage dependencies, significant human oversight will remain essential in computational biology workflows.

For the field, GeneBench-Pro provides a shared, credible standard for tracking progress on long-horizon biological reasoning—an area where current model performance has been difficult to measure rigorously.