MiMo-V2.5-Pro
Open-source MoE language model with 1M token context
Xiaomi released MiMo-V2.5-Pro, an open-source mixture of experts model with 1.02 trillion total parameters. It supports a one million token context window and targets complex software engineering and agentic tasks through hybrid attention architecture.
- 1.02 trillion total parameters with 42 billion active
- Supports context windows up to 1 million tokens
- Trained on 27 trillion tokens using FP8 precision
- Uses hybrid sliding window and global attention
- Open-source weights available on Hugging Face
What it is
MiMo-V2.5-Pro is a large language model developed by Xiaomi. It operates as a mixture of experts system, meaning it activates only a subset of its parameters for each task to improve efficiency. The model contains 1.02 trillion total parameters but uses just 42 billion during inference. It is designed for demanding applications such as complex software engineering and long-horizon agentic workflows. The system supports a context window of up to one million tokens, allowing it to process vast amounts of information in a single pass.
How it came about
This model builds upon the architecture introduced in MiMo-V2-Flash. It represents the most capable iteration in the MiMo series to date. The development focused on enhancing performance for tasks requiring sustained coherence over thousands of tool calls. The release includes both a base version and a pro version, with the latter optimised for instruction following and complex trajectories through post-training techniques.
Inside the model
The architecture interleaves sliding window attention with global attention in a six to one ratio. This hybrid approach reduces key-value cache storage by nearly seven times while maintaining long-context performance via learnable attention sink bias. It employs three lightweight multi-token prediction modules to triple output speed during inference. Training involved 27 trillion tokens using FP8 mixed precision. Post-training used supervised fine-tuning, domain-specialised reinforcement learning, and multi-teacher on-policy distillation to integrate capabilities from various expert teachers.
What it can do
The model demonstrates strong performance in general knowledge benchmarks such as MMLU and BBH. It shows particular strength in mathematics, scoring 99.6 on GSM8K and 86.2 on MATH. In coding tasks, it achieves 75.6 on HumanEval+ and 39.6 on LiveCodeBench v6. Long-context evaluation on GraphWalks shows it maintains reasoning ability at 512k tokens, though scores decline at the full one million token limit compared to shorter contexts. It handles Chinese language tasks effectively, scoring above 90 on C-Eval and CMMLU.
How to use it
The model weights are available openly on Hugging Face and ModelScope. It is distributed under an open-source licence that permits use with specific restrictions. Users are prohibited from employing the model for unlawful purposes, military applications, or malicious cyber activities. Deployment guides are provided for SGLang and vLLM frameworks to ensure optimal performance. The company recommends specific sampling parameters for local deployment.
Safety and testing
Post-training included domain-specialised training with safety teacher models. The usage licence explicitly forbids using the model to infringe on intellectual property rights, privacy rights, or to harm individuals. It prohibits engagement in malware creation, phishing, or unauthorised access to computer systems. Users are held solely responsible for any outputs or consequences arising from their use of the model.
Questions people ask about MiMo-V2.5-Pro
How many parameters does MiMo-V2.5-Pro have?
The model has 1.02 trillion total parameters. However, it is a mixture of experts model, so it only activates 42 billion parameters for each token during inference. This design allows for high capacity while maintaining computational efficiency.
What is the maximum context length?
MiMo-V2.5-Pro supports a context window of up to one million tokens. This allows it to process extremely long documents or maintain coherence over extended conversations and complex multi-step tasks without losing track of earlier information.
Is this model open source?
Yes, the model weights are openly available on platforms like Hugging Face and ModelScope. It is released under an open-source licence, though users must agree to restrictions prohibiting military use, illegal activities, and malicious cyber operations.
What architecture does it use?
It uses a hybrid attention architecture that combines sliding window attention and global attention. It also features multi-token prediction modules to accelerate inference. The model consists of 70 layers, with one dense layer and 69 mixture of experts layers.
Good reading and sources
There is no English Wikipedia article on MiMo-V2.5-Pro yet, so the facts on this page come from the sources listed above: huggingface.co/XiaomiMiMo/MiMo-V2.5-Pro. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.