Qwen3-Omni
Multimodal model processing text, audio, video and images with real-time speech output.
Qwen3-Omni is a multimodal foundation model released in September 2025 by the Qwen team. It processes text, images, audio and video to deliver streaming text and speech responses. The model uses a mixture of experts architecture to maintain high performance across all modalities without degrading unimodal capabilities.
- Supports 119 text languages and 19 speech input languages.
- Achieves open-source SOTA on 32 of 36 audio/video benchmarks.
- Uses Thinker-Talker MoE architecture for real-time streaming.
- Releases under Apache 2.0 licence with full model weights.
- Theoretical first-packet latency of 234 ms in cold-start settings.
What it is
Qwen3-Omni is a natively end-to-end multimodal foundation model developed by the Qwen team. Released in September 2025, it accepts text, images, audio and video as inputs. The system generates responses in both text and natural speech through real-time streaming. It supports interaction in 119 text languages, understands speech in 19 languages and produces speech output in 10 languages. The model aims to provide state-of-the-art performance across all modalities without sacrificing capability in any single area.
How it came about
The Qwen team introduced Qwen3-Omni in a technical report published on arXiv in September 2025. The developers focused on creating a single model that handles multiple input types simultaneously. They addressed the lack of general-purpose audio captioning tools in the open-source community by releasing a specialised fine-tuned variant. The project builds upon earlier work in the Qwen series, extending its capabilities to include native audio and video processing alongside traditional text and image tasks.
Inside the model
The model employs a Thinker-Talker mixture of experts architecture. This design separates perception and generation tasks to improve efficiency. A multi-codebook scheme allows the Talker component to predict discrete speech codecs autoregressively. This approach replaces heavier block-wise diffusion methods with a lightweight causal ConvNet. The change enables streaming from the first codec frame, achieving a theoretical first-packet latency of 234 milliseconds in cold-start settings. Early text-first pretraining and mixed multimodal training provide strong general representations.
What it can do
Qwen3-Omni achieves state-of-the-art results on 22 of 36 audio and video benchmarks. It holds the open-source state-of-the-art on 32 of these benchmarks. Performance in automatic speech recognition and voice conversation compares favourably to closed-source models like Gemini 2.5 Pro. The model supports complex tasks such as solving mathematical problems from images, analysing music styles and generating navigation commands from first-person video. A dedicated Thinking model variant provides chain-of-thought reasoning for any input modality.
How to use it
The weights for Qwen3-Omni-30B-A3B-Instruct, Qwen3-Omni-30B-A3B-Thinking and Qwen3-Omni-30B-A3B-Captioner are available under the Apache 2.0 licence. Users can download them via Hugging Face or ModelScope. Running the model requires installing specific dependencies, including FlashAttention 2 for efficient memory usage. The code has been merged into Hugging Face Transformers, though users may need to install from source until a PyPI package is released.
Why it matters
The release fills a critical gap in the open-source community by providing a high-quality audio captioning model. By demonstrating that a single model can handle text, image, audio and video without performance regression, it sets a new standard for multimodal systems. The low-latency streaming capability enables more natural human-computer interaction through voice.
Questions people ask about Qwen3-Omni
What input types does Qwen3-Omni accept?
The model processes text, images, audio and video inputs. It can handle interleaved combinations of these modalities, allowing for complex queries that involve seeing an image while hearing audio or watching a video clip.
How many languages does it support for speech?
Qwen3-Omni understands speech input in 19 languages, including English, Chinese, Japanese and Arabic. It can generate speech output in 10 languages, covering major global languages such as English, Spanish, French and German.
Is the model available for local use?
Yes, the model weights are open source under the Apache 2.0 licence. You can download them from Hugging Face or ModelScope and run them locally using Hugging Face Transformers or vLLM, provided you have compatible GPU hardware.
What makes its speech output fast?
It uses a multi-codebook design where the Talker component predicts discrete speech codecs. This replaces computationally heavy diffusion methods with a lightweight causal ConvNet, enabling streaming from the very first frame with low latency.
Good reading and sources
There is no English Wikipedia article on Qwen3-Omni yet, so the facts on this page come from the sources listed above: huggingface.co/Qwen/Qwen3-Omni-30B-A3B-Instruct, arxiv.org/abs/2509.17765. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.