Databricks introduces OfficeQA Pro V2, a harder benchmark for enterprise grounded reasoning

Databricks has released OfficeQA Pro V2, a benchmark built to test whether AI agents can generalize to unfamiliar, multi-document enterprise research tasks rather than the narrower single-document setups most benchmarks use.

What's new

The benchmark poses 90 questions grounded in roughly 120,000 pages drawn from the U.S. Treasury's Accounts of Receipts and Expenditures — a dataset the Treasury released as a cohesive collection for the first time to mark the 250th anniversary of the United States. Databricks built it as the benchmark for the inaugural Databricks Grounded Reasoning Cup, a live competition in which 11 academic teams, supported by OpenAI, Anthropic, and Google DeepMind, built agents evaluated on a previously unseen corpus.

Compared with the earlier OfficeQA Pro benchmark, which drew on about 2 source documents per question on average, OfficeQA Pro V2 questions require evidence from 6.7 source documents on average (median 5.5, maximum 24), and 74.4% of questions need four or more sources — up from 62.4% in OfficeQA Pro.

Out-of-the-box frontier agents — Claude Code running Claude Opus 4.8 and Claude Fable 5, and Codex running GPT-5.5 and GPT-5.6 Sol — averaged 37.5% accuracy on the new benchmark. Agents built specifically for the Grounded Reasoning Cup did better, averaging 41.1%, with the winning team reaching 63.3%. Databricks' own Genie agent, using the same underlying models plus Databricks' ai_parse tool to pre-parse the document corpus, improved accuracy by an average of 24.0 percentage points over the default harnesses, with its strongest configuration reaching 60% — a 15.3-point jump from 37.5% to 52.8% across the four directly matched models. Databricks summarized the results: "grounded reasoning remains far from solved, but that the right agent harness can unlock significant gains from existing frontier models."

Context

OfficeQA and its harder OfficeQA Pro subset have become a widely cited measure of frontier model and agent capability on document retrieval and analytical reasoning since their introduction. But Databricks says that progress raised a question its own researchers wanted to test directly: whether gains on that benchmark reflected genuine improvements in grounded reasoning, or progress specific to one corpus and task distribution. OfficeQA Pro V2's entirely new document set and higher source-document count are designed to probe that distinction.

Why it matters

The gap between the 37.5% out-of-the-box baseline and Genie's 52-60% range is a concrete illustration that a meaningful share of the difference between AI demos and production-ready enterprise deployments isn't raw model capability — it's the surrounding retrieval and reasoning harness built around the model. For enterprise buyers evaluating agents for document-heavy workflows like compliance, finance, or legal research, that gap is arguably a more useful signal than headline model benchmark scores, and it gives competing vendors a concrete, freshly released target to beat.