News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
imgwavvidQwen2.5-Omni

Qwen2.5-Omni

End-to-end multimodal model for real-time speech and video chat.

Qwen2.5-Omni is an end-to-end multimodal language model released in March 2025 by the Qwen Team at Alibaba Group. It processes text, images, audio, and video to generate simultaneous text and speech responses. The model uses a novel Thinker-Talker architecture to handle streaming inputs and outputs with low latency.

Share on WhatsApp

  • Uses Thinker-Talker architecture for simultaneous text and speech.
  • Introduces TMRoPE for syncing video and audio timestamps.
  • Achieves state-of-the-art performance on Omni-Bench.
  • Supports real-time streaming for voice and video chat.
  • Available in 7B and 3B parameter sizes with open weights.

What it is

Qwen2.5-Omni is an end-to-end multimodal model developed by the Qwen Team at Alibaba Group. Released in March 2025, it accepts text, images, audio, and video as inputs. The system generates both text and natural speech responses simultaneously in a streaming manner. It is designed for real-time interactions, supporting chunked input and immediate output. The model comes in 7-billion and 3-billion parameter sizes. It aims to provide robust performance across diverse modalities while maintaining low latency for voice and video chat applications.

How it came about

The model was announced in a technical report published on arXiv in March 2025. It builds upon previous iterations in the Qwen family, specifically surpassing the capabilities of Qwen2-Audio in audio tasks. The developers positioned it as comparable in size and performance to Qwen2.5-VL. The release included open weights hosted on Hugging Face, allowing researchers and developers to access the model directly. The team highlighted its ability to match text-based instruction following when using speech inputs.

Inside the model

The core design features a Thinker-Talker architecture. The Thinker component acts as a large language model responsible for text generation. The Talker is a dual-track autoregressive model that uses hidden representations from the Thinker to produce audio tokens. This setup allows concurrent generation of text and speech without interference. To handle multimodal inputs, the model uses block-wise processing for audio and visual encoders. A novel position embedding called TMRoPE synchronizes timestamps between video and audio inputs. A sliding-window DiT mechanism reduces initial package delay during audio decoding.

What it can do

Qwen2.5-Omni achieves state-of-the-art results on the Omni-Bench multimodal benchmark. In speech instruction following, its performance on MMLU and GSM8K benchmarks rivals its text-based capabilities. The model outperforms Qwen2-Audio in audio tasks and matches Qwen2.5-VL-7B in visual reasoning. It shows strong results in speech recognition on Common Voice and translation on CoVoST2. For speech generation, the streaming Talker demonstrates superior robustness and naturalness compared to many existing alternatives. It also performs well in video understanding tasks such as MVBench and EgoSchema.

How to use it

The model weights are available openly via Hugging Face. Users can run the model using the Hugging Face Transformers library, with specific support added in version 4.51.3. The developers provide a utility toolkit called qwen-omni-utils to handle various input types, including base64, URLs, and interleaved media. Installation requires ffmpeg for audio and video processing. The code supports flash attention for better acceleration and memory efficiency. Both 7-billion and 3-billion parameter versions are accessible for local deployment.

Questions people ask about Qwen2.5-Omni

What makes Qwen2.5-Omni different from previous models?

It uses a new Thinker-Talker architecture that allows it to generate text and speech at the same time. It also introduces TMRoPE, a method to synchronize audio and video inputs precisely. This enables true end-to-end streaming for real-time conversations.

Can it understand video and audio together?

Yes, the model perceives text, images, audio, and video simultaneously. It uses block-wise processing for encoders and TMRoPE to align timestamps. This allows it to handle complex multimodal inputs like videos with spoken dialogue effectively.

How do I run this model locally?

You can download the weights from Hugging Face and use the Transformers library. You need to install qwen-omni-utils for handling media inputs. Ensure you have ffmpeg installed on your system for audio and video processing support.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Qwen2.5-Omni yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2503.20215, huggingface.co/Qwen/Qwen2.5-Omni-7B. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.