Together AI: cascading DeepSeek V4 Pro into GPT-5.6 Sol beats either model alone on cost and accuracy

Together AI published a benchmark comparing DeepSeek V4 Pro (the August 13 update) against OpenAI's GPT-5.6 Sol on DeepSWE, a software-engineering task suite, and found that routing between the two models beats using either one alone.

What's new

Per Together AI's blog post: "Run DeepSeek V4 Pro 0813 first, escalate to GPT-5.6 Sol when the tests fail. That cascade solves 83.0% of DeepSWE tasks at $3.35 each. Sol alone solves 72.7% at $8.37. Ten points better, 60% cheaper."

The specific numbers Together AI reports:

• Single-attempt accuracy (pass@1): GPT-5.6 Sol scores 72.7% (±2.2%), versus 62.8% (±3.1%) for DeepSeek V4 Pro 0813 — Sol holds the lead through pass@2 as well (81.0% vs 78.5%).
• Cost per rollout: Sol costs $8.37 versus Pro's $0.24 — a 35x price gap. Per $100 spent, Together AI calculates Pro solves 261 tasks to Sol's 9.
• Best-of-four accuracy (pass@4): DeepSeek V4 Pro overtakes Sol, 88.5% versus 85.8%, once given multiple attempts.
• Cascade routing: Running Pro first and escalating to Sol only when a test suite rejects Pro's answer reaches 83.0% accuracy at $3.35 per task — beating Sol's standalone accuracy while costing about 60% less.
• Failure characteristics: Together AI notes Sol's failures are "messier" — 20% of them break tests that were already passing, versus 11% for Pro — and recommends gating Sol's output on regressions specifically.

Context

The comparison lands a week after DeepSeek-V4-Pro reached general availability with major agent-benchmark gains, and days after OpenAI's Ultrafast mode shipped for GPT-5.6 Sol. Together AI, which sells inference for open models like DeepSeek's alongside routing and orchestration tooling across providers, has a direct commercial interest in showing that model cascades beat single-model deployments — it's the core pitch behind offering DeepSeek-V4 Pro 0813 on its own US-hosted infrastructure.

Why it matters

The result quantifies a pattern that's become common as the price gap between frontier and open models widens: a cheap model attempting first, with an expensive model as fallback only when verification fails, can match or beat the expensive model's own accuracy at a fraction of the cost. That only works when there's a reliable pass/fail signal — a test suite, in DeepSWE's case — to gate the escalation, which limits the approach to verifiable domains like coding rather than open-ended tasks. For teams building coding agents, it's a concrete data point favoring routing architectures over defaulting to the single most capable, most expensive model for every request.