NVIDIA releases NemotronLabs VoiceChat 11B, the first open full-duplex speech model with live tool calling

NVIDIA has published NemotronLabs VoiceChat, an 11-billion-parameter end-to-end speech model the company describes as the first open full-duplex voice model that can call tools mid-conversation, according to the model card on Hugging Face.

What's new

• The model card states: "NVIDIA NemotronLabs VoiceChat is the first open full-duplex model to support tool calling while maintaining a natural conversation flow during tool execution."
• It fuses a fast Conformer speech encoder, a Nemotron Nano V2 9B LLM backbone, and a TTS decoder into one unified architecture, rather than chaining separate ASR, LLM, and TTS models together.
• NVIDIA reports roughly 450ms turn-taking latency, natural interruption and barge-in handling, and a developer-defined "on-hold" spoken message that plays while a tool call executes.
• On NVIDIA's own benchmarking, the model ranks #2 on VoiceBench and #2 on Full-Duplex-Bench 1.0 among open full-duplex models.
• Audio input runs at 16 kHz and output at 22.05 kHz. Weights and code, hosted on GitHub under NVIDIA-NeMo/Speech, are released under the OpenMDW License Agreement, version 1.1; the card notes the model "is ready for research purposes only."

Context

VoiceChat is part of NVIDIA's Nemotron Labs initiative, which has spent recent months shipping a family of open Nemotron models aimed at agentic and multimodal workloads, including Nemotron 3 Super and Nemotron 3 Nano Omni. Full-duplex speech, where a model listens and speaks at the same time and can be interrupted mid-sentence rather than waiting for a scripted turn, has generally lagged in open-source releases compared with closed, proprietary voice stacks from labs like OpenAI and Google.

Why it matters

Tool calling is what turns a voice model from a conversational demo into something that can actually get work done — booking something, looking something up, triggering an action — without breaking the natural back-and-forth of speech. NVIDIA's claim to be the first open model that does this while staying full-duplex, if it holds up under independent testing, gives developers building voice agents a self-hostable alternative to the closed voice stacks at OpenAI, Google, and xAI, all three of which shipped voice updates of their own this same week. Because the weights are open and hosted on Hugging Face, teams can inspect, fine-tune, or run the model on their own infrastructure — something none of the closed frontier labs' voice products currently allow.