Hunyuan-TurboS
Hybrid Mamba-Transformer MoE model by Tencent
Tencent Hunyuan Team released Hunyuan-TurboS in May 2025. It is a large hybrid model combining Mamba and Transformer architectures with a Mixture of Experts design. The system uses an adaptive chain-of-thought mechanism to balance speed and deep reasoning for complex tasks.
- 560 billion total parameters with 56 billion active
- First industry-deployed large-scale Mamba model
- Supports 256K context length
- Scored 1356 on LMSYS Chatbot Arena
- Trained on 16 trillion tokens
What it is
Hunyuan-TurboS is a large language model developed by the Tencent Hunyuan Team. Announced in May 2025, it represents a shift towards hybrid architectures in industrial applications. The model contains 560 billion total parameters, with 56 billion active during inference due to its Mixture of Experts structure. It is designed to handle long sequences efficiently while maintaining strong contextual understanding through a combination of Mamba and Transformer components.
How it came about
The Tencent Hunyuan Team introduced this model in a paper submitted to arXiv in May 2025. The development involved a large group of researchers and engineers. It marks the first industry-deployed large-scale model based on the Mamba architecture. The team focused on creating a system that could be deployed practically while offering high performance, revising the initial submission in July 2025.
Inside the model
The architecture blends Mamba2 blocks for linear complexity and speed with Transformer attention mechanisms for better context handling. It uses Grouped-Query Attention to reduce memory usage in the key-value cache. The feed-forward networks employ a Mixture of Experts approach. The model has 128 layers arranged in an innovative AMF/MF block pattern. Training involved 16 trillion high-quality tokens. Post-training included supervised fine-tuning on three million instructions, adaptive chain-of-thought fusion, multi-round deliberation learning, and two-stage reinforcement learning focused on STEM and general instruction following.
What it can do
The model supports a context window of 256,000 tokens. It features an adaptive long-short chain-of-thought mechanism that switches between quick answers for simple queries and deeper thinking modes for complex problems. On the LMSYS Chatbot Arena, it achieved a score of 1356, ranking in the top seven overall. This performance exceeded Gemini-2.0-Flash-001 and o4-mini-2025-04-16. It averaged 77.9 per cent across 23 automated benchmarks, balancing high capability with lower inference costs than many dedicated reasoning models.
How to use it
The model is described as industry-deployed, suggesting availability through Tencent's services or APIs. The technical report details its architecture and training methods but does not specify open weights or public download options. It is positioned as a practical solution for commercial use cases requiring efficiency and scale.
Why it matters
Hunyuan-TurboS establishes a new paradigm for efficient large-scale pre-trained models by proving the viability of Mamba-Transformer hybrids in production. Its success demonstrates that combining state-space models with transformers can yield significant computational savings without sacrificing performance. This approach may influence future designs where inference cost and sequence length are critical constraints.
Questions people ask about Hunyuan-TurboS
What makes Hunyuan-TurboS different from other models?
It combines Mamba and Transformer architectures in a hybrid design. This allows it to process long sequences efficiently using Mamba while retaining the strong contextual understanding of Transformers. It also uses a Mixture of Experts structure to activate only a fraction of its parameters during use.
How does the adaptive chain-of-thought work?
The model dynamically switches between rapid response modes for simple questions and deep thinking modes for complex problems. This optimises computational resources by avoiding unnecessary processing for easy tasks while ensuring thorough reasoning for difficult ones.
How large is the model?
Hunyuan-TurboS has 560 billion total parameters. However, due to its Mixture of Experts design, only 56 billion parameters are activated for any given token. This reduces the computational cost during inference compared to dense models of similar total size.
Good reading and sources
There is no English Wikipedia article on Hunyuan-TurboS yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2505.15431. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.