NVIDIA releases Nemotron 3 Ultra Teacher-Chat, a 550B open-weight model for distillation and preference data

NVIDIA has published Nemotron 3 Ultra Teacher-Chat, an open-weight conversational model designed less for direct chatbot use than as a "teacher" for training smaller models — generating conversational data, preference pairs, and quality-graded responses that other teams can use to fine-tune their own systems.

What's new

Per NVIDIA's model card on Hugging Face, Teacher-Chat is "a specialized conversational model in the Nemotron 3 Ultra family," built on a Latent Mixture-of-Experts (LatentMoE) architecture that interleaves Mamba-2 and MoE layers with select attention layers, plus multi-token prediction layers for faster inference. The model has 550B total parameters with 55B active at a time, supports context windows up to 1M tokens, and covers ten languages including English, French, Spanish, Italian, German, Japanese, Korean, Hindi, Brazilian Portuguese, and Chinese. It ships under NVIDIA's OpenMDW-1.1 license, which the card describes as ready for both commercial and non-commercial use, and pretraining drew on roughly 20 trillion tokens, including NVIDIA's Nemotron-CC-v2/v2.1 web corpus and its Nemotron-Pretraining-Code datasets.

Unlike a general-purpose assistant, Teacher-Chat is explicitly positioned as a domain-specialized teacher for what NVIDIA calls Multi-Teacher On-Policy Distillation (MOPD) — pairing with more than ten other domain-specialized teacher models to generate training signal for smaller downstream models. Its stated use cases are conversational data generation, preference-pair creation, and response quality grading, with configurable reasoning traces that can be toggled through chat templates. Minimum deployment hardware is steep: NVIDIA lists 4x B200/GB200/GB300 GPUs for single-node use, or 8x H100s, reflecting the model's scale.

Context

Teacher-Chat extends NVIDIA's Nemotron 3 Ultra line, which the company has been building out through 2026 as part of its "Nemotron Labs" push toward fully open, inspectable models — open weights, open training data recipes, and open techniques — aimed at enterprises and governments that want control over the models they deploy rather than closed APIs. It follows other recent Nemotron releases this month, including efficiency-focused Nemotron 3.5 Lightning variants aimed at agentic workloads, and sits alongside NVIDIA's embedding-model releases in the same family.

Why it matters

Rather than compete directly as a chat assistant, Teacher-Chat targets the model-training supply chain itself: any lab doing distillation, RLHF-style preference data collection, or synthetic conversational data generation now has a fully open, commercially licensed 550B-parameter teacher model to draw on instead of building one from scratch or relying on a closed API for that role. That's a meaningful lower-cost, more controllable alternative for teams building their own smaller specialized models, and it reinforces NVIDIA's strategy of open-sourcing infrastructure-layer models to seed demand for its own hardware.