NVIDIA serves Alibaba's 2.4-trillion-parameter Qwen3 model on GB300 NVL72 at Day 0

NVIDIA has published deployment guidance and Day 0 performance numbers for running Alibaba's Qwen3.8-2.4T-A95B — a 2.4-trillion-parameter mixture-of-experts model — on its GB300 NVL72 rack-scale system.

What's new

The Qwen3.8-2.4T-A95B model uses a fine-grained mixture-of-experts architecture with 95 billion parameters activated per token, combined with hybrid full and linear attention layers. NVIDIA describes the design's purpose directly: fine-grained MoE "makes the 2.4T parameter count practical to serve...delivering frontier-scale capacity at a fraction of the cost of a comparable dense model." The model supports context windows up to 1 million tokens and output lengths up to 128K tokens, aimed at agentic tasks such as coding and long-document analysis, with built-in reasoning controls — low, high, and xhigh — that let developers trade inference compute for reasoning depth per request.

On the hardware side, NVIDIA reports the model running at "over 4,000 tokens per second per GPU and over 350 tokens per second per user on NVIDIA GB300 NVL72 in FP8 precision on Day 0." The GB300 NVL72 rack pairs 72 Blackwell Ultra GPUs over 130 TB/s of NVLink bandwidth, treating the rack as a single accelerator for a model too large to fit on any individual GPU.

Context

Qwen3.8-2.4T-A95B is the largest model yet in Alibaba's Qwen3 line, which has otherwise centered on much smaller dense and MoE variants (including a 27B model that launched on Cerebras hardware). Serving a model at this parameter count depends on exactly the kind of high-bandwidth, multi-GPU rack NVIDIA is describing here — without NVLink-class interconnect, a 2.4T-parameter MoE would have to shard across nodes with far more communication overhead. NVIDIA says it is extending that work with "optimized kernels, inference runtimes, and distributed serving recipes" for multinode deployments beyond a single rack.

Why it matters

Day-0 vendor-validated performance numbers for a frontier-scale open-weight model are a signal of how NVIDIA is positioning GB300 against the next generation of very large MoE models coming out of Chinese labs. For Alibaba, having NVIDIA publish tuned serving numbers on its newest hardware the same day the model ships is validation that Qwen3's largest variant is viable to run at production token-per-second rates, not just a research release — a meaningful data point for any team weighing self-hosted open-weight models against closed frontier APIs at this scale.