News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
fnQwen3-Next-80B-A3B

Qwen3-Next-80B-A3B

An 80-billion parameter hybrid model with ultra-long context support.

Qwen3-Next-80B-A3B is a large language model developed by the Qwen team. It uses a hybrid architecture to handle contexts up to one million tokens efficiently. The model activates only three billion parameters per token, balancing speed and capacity for agentic tasks.

Share on WhatsApp

  • Activates only 3 billion of 80 billion parameters per token.
  • Supports context windows up to 1,010,000 tokens.
  • Uses hybrid attention with Gated DeltaNet and Gated Attention.
  • Trained on 15 trillion tokens for robust performance.
  • Outperforms Qwen3-32B-Base with 10% of training cost.

What it is

Qwen3-Next-80B-A3B is the first model in the Qwen3-Next series. It features eighty billion total parameters but activates only three billion during inference. This design aims to improve scaling efficiency for artificial intelligence agents. The model supports native context lengths of over two hundred thousand tokens. It can extend this window to more than one million tokens. The developers state it offers strong performance in parameter efficiency and inference speed compared to earlier versions.

How it came about

The Qwen team developed this model to address demands for longer contexts and larger parameter counts. They observed trends toward scaling both metrics for more powerful agentic AI. Qwen3-Next-80B-A3B represents their latest advancement in architectural efficiency. It follows previous iterations in the Qwen3 family. The release includes both base and instruct variants. The instruct version is designed for non-thinking modes and does not generate specific reasoning blocks.

Inside the model

The architecture combines Gated DeltaNet and Gated Attention mechanisms. This hybrid layout replaces standard attention methods to model ultra-long contexts efficiently. It uses a high-sparsity mixture of experts approach. Only ten experts are activated per token, which drastically reduces computational cost. Stability optimizations include zero-centered layernorm and weight decay. Multi-token prediction boosts pretraining performance and accelerates inference. The model was trained on fifteen trillion tokens.

What it can do

The instruct variant performs on par with much larger models on certain benchmarks. It shows advantages in handling ultra-long-context tasks. Scores include 80.6 on MMLU-Pro and 90.9 on MMLU-Redux. It achieves 75.8 on LiveBench and 56.6 on LiveCodeBench v6. The model demonstrates strong multilingual capabilities across various tests. It excels in tool calling and agentic workflows when used with compatible frameworks.

How to use it

The model weights are available on Hugging Face. Developers can use libraries like SGLang or vLLM for deployment. These tools allow creation of OpenAI-compatible API endpoints. Specific versions of these libraries are required for full functionality. Multi-token prediction is not generally available in standard transformers. Dedicated inference frameworks are recommended for optimal throughput and efficiency.

Questions people ask about Qwen3-Next-80B-A3B

How many parameters does Qwen3-Next-80B-A3B have?

The model has eighty billion total parameters. However, it uses a sparse mixture of experts architecture. This means only three billion parameters are activated for each token during inference.

What is the maximum context length supported?

The model supports a native context length of 262,144 tokens. It is extensible up to 1,010,000 tokens. This allows it to process extremely long documents or codebases efficiently.

Which inference frameworks are recommended?

SGLang and vLLM are recommended for deployment. They support OpenAI-compatible APIs and optimize throughput. Standard Hugging Face transformers may not fully support multi-token prediction features.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Qwen3-Next-80B-A3B yet, so the facts on this page come from the sources listed above: huggingface.co/Qwen/Qwen3-Next-80B-A3B-Instruct. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.