News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
Qwen2

Qwen2

Open-weight language model series from Alibaba

Qwen2 is a family of large language models released by Alibaba in July 2024. It includes dense and mixture-of-experts architectures with sizes up to 72 billion parameters. The models support multilingual tasks, coding, and reasoning, offering open weights for research and deployment.

Share on WhatsApp

  • Supports context length up to 131,072 tokens
  • Includes models from 0.5B to 72B parameters
  • Proficient in approximately 30 languages
  • Uses YARN for long text processing
  • Open weights available on Hugging Face

What it is

Qwen2 is a series of large language models developed by Alibaba. Released in mid-2024, the family ranges from small 0.5 billion parameter models to a flagship 72 billion parameter version. It includes both dense models and a mixture-of-experts variant. The system handles text generation, coding, mathematics, and reasoning tasks. It supports approximately thirty languages, including English, Chinese, Arabic, and Japanese. The models are designed for broad accessibility in research and commercial applications.

How it came about

This series follows the Qwen1.5 models. The technical report was published on arXiv in July 2024, with revisions continuing into September. The developers aimed to surpass previous open-weight models and compete with proprietary systems. The release included base models and instruction-tuned variants for chat applications. Weights were made available on platforms like Hugging Face and ModelScope to encourage community use.

Inside the model

The architecture uses a decoder-only transformer design. Key features include SwiGLU activation functions, attention QKV bias, and group query attention. An improved tokenizer adapts to multiple natural languages and code. Training involved pretraining on large datasets, followed by supervised fine-tuning. Direct preference optimization was used for alignment. For long contexts, the model employs YARN technique to handle inputs up to 131,072 tokens.

What it can do

The 72 billion parameter base model scores 84.2 on MMLU and 64.6 on HumanEval. The instruction-tuned variant achieves 9.1 on MT-Bench and 35.7 on LiveCodeBench. It demonstrates strong performance in multilingual proficiency and coding tasks. Benchmarks show it outperforms many prior open models in language understanding and generation. Mathematical reasoning scores include 89.5 on GSM8K for the base model.

How to use it

Model weights are openly available on Hugging Face and ModelScope. Supplementary materials and example code are hosted on GitHub. Users can download weights for local deployment or fine-tuning. The code integrates with Hugging Face transformers library version 4.37.0 or higher. Deployment tools like vLLM are recommended for handling long context windows efficiently.

Questions people ask about Qwen2

What is the maximum context length for Qwen2?

The instruction-tuned 72B model supports a context length of up to 131,072 tokens. This allows processing of extensive inputs using the YARN technique for length extrapolation.

Is Qwen2 available for free download?

Yes, the model weights are openly available on Hugging Face and ModelScope. Supplementary materials and code are also provided on GitHub for research and deployment.

How many languages does Qwen2 support?

Qwen2 is proficient in approximately 30 languages. These include English, Chinese, Spanish, French, German, Arabic, Russian, Korean, Japanese, Thai, and Vietnamese.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Qwen2 yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2407.10671, huggingface.co/Qwen/Qwen2-72B-Instruct. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.