Together AI benchmark: GLM-5.3 solves coding tasks nearly twice as cheap as GPT-5.6 Sol

Together AI ran a head-to-head coding benchmark pitting Zhipu's GLM-5.3 against OpenAI's GPT-5.6 Sol, finding that while Sol edges out GLM-5.3 on raw accuracy, GLM-5.3 solves far more tasks per dollar — and a simple cascade strategy that escalates only failed attempts to Sol beats both models running alone.

What's new

The test used DeepSWE v1.1, a 113-task software engineering benchmark, run across 4 trials per task for 904 total rollouts. The headline numbers:

• Pass@1 (single attempt): GPT-5.6 Sol scored 72.7% against GLM-5.3's 69.0%. As Together AI put it, "Sol wins the single shot, narrowly. 72.7% pass@1 against GLM-5.3's 69.0%, a 3.7 point gap that sits inside a couple of standard deviations."
• Pass@4 (best of four attempts): GLM-5.3 actually pulled ahead, hitting 87.6% versus Sol's 85.8%.
• Cost per rollout: GLM-5.3 ran $3.99 per rollout compared to Sol's $8.37 — a 2.1x gap. "The price gap is 2.1x. $3.99 per rollout against $8.37. Per $100 spent, GLM-5.3 solves 17 tasks and Sol solves 9."
• Cascade routing: Running GLM-5.3 first and escalating only rejected answers to Sol produced 85.9% accuracy at $6.61 per task — beating GLM-5.3 alone on accuracy while still costing less than half of running Sol on everything.

Context

GLM-5.3 is Zhipu AI's latest release in the GLM family, one of several Chinese open-model efforts (alongside DeepSeek and Qwen) that have closed the gap with U.S. frontier labs on coding benchmarks over the past year. GPT-5.6 Sol is OpenAI's current flagship coding-oriented model, recently the subject of its own pricing cut and an Ultrafast serving tier. Together AI, an inference platform that hosts both open and proprietary models, has increasingly published these kinds of head-to-head cost/accuracy benchmarks as a way to help developers pick a model-routing strategy rather than a single "best" model.

Why it matters

The result reinforces a pattern showing up across coding benchmarks this year: frontier closed models retain a small accuracy edge in a single attempt, but that edge shrinks or disappears once you allow multiple attempts or verification, and it's swamped by the cost difference at scale. For teams running agentic coding pipelines with automated test verification, a cascade — cheap model first, escalate on failure — can now beat either model running alone on both cost and accuracy. That's a meaningful signal for anyone designing model-routing logic into coding agents rather than hard-coding a single provider.