NVIDIA releases Nemotron 3 Ultra, a 550B open-weights MoE reasoning model with sub-quadratic scaling and 1M-token context for long-running agents

NVIDIA released Nemotron 3 Ultra on June 4, 2026 — a 550 billion total parameter, 55 billion active parameter open-weights model built for long-horizon agent workloads. The model uses a hybrid Mamba-Attention architecture designed to handle very long contexts at speeds well above comparable-size open models.

What's new

Key specifications:

• Architecture: Mixture-of-Experts Hybrid Mamba-Transformer with MTP layers for speculative decoding
• Parameters: 550B total, 55B active (sparse MoE)
• Context window: 1 million tokens
• Inference throughput: 5.9x faster than GLM-5.1-754B, 4.8x faster than Kimi-K2.6-1T, 1.6x faster than Qwen-3.5-397B on comparable hardware
• Benchmark result: Scores 48 on the Artificial Analysis Intelligence Index, ranking as the most capable open model from a US lab at launch
• License: OpenMDW-1.1 (permissive open weights)
• Checkpoints: Pre-trained, post-trained, and NVFP4-quantized (optimized for Blackwell, Hopper, and Ampere GPUs)
• Availability: Hugging Face, ModelScope, and OpenRouter; training datasets and model recipes on GitHub

The Mamba layers handle long sequences with sub-quadratic scaling — meaning memory and compute requirements grow more slowly than standard full-attention models as context length increases. A small number of standard Attention layers are retained for tasks requiring precise recall across the full context.

Context

NVIDIA announced Nemotron 3 Ultra in Jensen Huang's Computex 2026 keynote on June 1, with weights released three days later. The Nemotron line is NVIDIA's open-weights research series, distinct from the commercial NVIDIA AI Enterprise stack. Previous releases include Nemotron-4 340B (2024) and models in the Minitron family.

The hybrid Mamba-Attention design places Nemotron 3 Ultra in a growing class of architectures — alongside xAI's and Mistral's similar experiments — that combine SSM-based long-range processing with attention-based precision to achieve better inference scaling than pure-transformer models at long context.

At 5.9x throughput versus GLM-5.1-754B, NVIDIA is effectively arguing that Nemotron 3 Ultra's 55B active parameters process tokens faster than GLM's much larger active parameter count — the sparse MoE design means most parameters are idle for any given token.

Why it matters

The open release matters in two dimensions. First, it provides a competitive open checkpoint for enterprise teams that need to run large-context agent workloads on-premise or in private cloud — sectors including finance, defense, and healthcare where model weights must stay within a controlled boundary. The OpenMDW-1.1 license and full training recipe release mean organizations can fine-tune and audit the model.

Second, NVIDIA distributing its own frontier open model directly competes with Meta's Llama 4, Mistral's open releases, and DeepSeek's V4 series — all of which target the same enterprise open-weights market. For NVIDIA, a widely adopted open model generates hardware pull: teams running Nemotron 3 Ultra at scale will route those workloads to Blackwell GPUs, the only hardware for which NVIDIA ships an optimized NVFP4 checkpoint.