Qwen2
Open-weight language model series from Alibaba
Qwen2 is a family of large language models released by Alibaba in July 2024. It includes dense and mixture-of-experts architectures with sizes up to 72 billion parameters. The models support multilingual tasks, coding, and reasoning, offering open weights for research and deployment.
- Supports context length up to 131,072 tokens
- Includes models from 0.5B to 72B parameters
- Proficient in approximately 30 languages
- Uses YARN for long text processing
- Open weights available on Hugging Face
What it is
Qwen2 is a series of large language models developed by Alibaba. Released in mid-2024, the family ranges from small 0.5 billion parameter models to a flagship 72 billion parameter version. It includes both dense models and a mixture-of-experts variant. The system handles text generation, coding, mathematics, and reasoning tasks. It supports approximately thirty languages, including English, Chinese, Arabic, and Japanese. The models are designed for broad accessibility in research and commercial applications.
How it came about
This series follows the Qwen1.5 models. The technical report was published on arXiv in July 2024, with revisions continuing into September. The developers aimed to surpass previous open-weight models and compete with proprietary systems. The release included base models and instruction-tuned variants for chat applications. Weights were made available on platforms like Hugging Face and ModelScope to encourage community use.
Inside the model
The architecture uses a decoder-only transformer design. Key features include SwiGLU activation functions, attention QKV bias, and group query attention. An improved tokenizer adapts to multiple natural languages and code. Training involved pretraining on large datasets, followed by supervised fine-tuning. Direct preference optimization was used for alignment. For long contexts, the model employs YARN technique to handle inputs up to 131,072 tokens.
What it can do
The 72 billion parameter base model scores 84.2 on MMLU and 64.6 on HumanEval. The instruction-tuned variant achieves 9.1 on MT-Bench and 35.7 on LiveCodeBench. It demonstrates strong performance in multilingual proficiency and coding tasks. Benchmarks show it outperforms many prior open models in language understanding and generation. Mathematical reasoning scores include 89.5 on GSM8K for the base model.
How to use it
Model weights are openly available on Hugging Face and ModelScope. Supplementary materials and example code are hosted on GitHub. Users can download weights for local deployment or fine-tuning. The code integrates with Hugging Face transformers library version 4.37.0 or higher. Deployment tools like vLLM are recommended for handling long context windows efficiently.
Questions people ask about Qwen2
What is the maximum context length for Qwen2?
The instruction-tuned 72B model supports a context length of up to 131,072 tokens. This allows processing of extensive inputs using the YARN technique for length extrapolation.
Is Qwen2 available for free download?
Yes, the model weights are openly available on Hugging Face and ModelScope. Supplementary materials and code are also provided on GitHub for research and deployment.
How many languages does Qwen2 support?
Qwen2 is proficient in approximately 30 languages. These include English, Chinese, Spanish, French, German, Arabic, Russian, Korean, Japanese, Thai, and Vietnamese.
Good reading and sources
There is no English Wikipedia article on Qwen2 yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2407.10671, huggingface.co/Qwen/Qwen2-72B-Instruct. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.