News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
Qwen2.5-72B

Qwen2.5-72B

A 72-billion parameter open-weight language model from Alibaba Cloud.

Alibaba Cloud released Qwen2.5-72B in September 2024 as part of its Qwen2.5 series. This dense decoder-only model features 72 billion parameters and supports long contexts up to 128K tokens. It offers strong performance in coding, mathematics, and multilingual tasks across more than 29 languages.

Share on WhatsApp

  • Trained on 18 trillion tokens of data
  • Supports context lengths up to 128K tokens
  • Licensed under Apache 2.0 for most variants
  • Strong performance in coding and math benchmarks
  • Supports over 29 languages including Chinese and English

What it is

Qwen2.5-72B is a large language model developed by Alibaba Cloud. It belongs to the Qwen2.5 family, which was announced in September 2024. The model contains 72 billion parameters and uses a dense decoder-only architecture. It is designed for general purpose tasks with specific strengths in coding and mathematics. The system supports context windows of up to 128,000 tokens and can generate outputs of up to 8,000 tokens. It handles text in over 29 languages, including English, Chinese, Japanese, and Arabic.

How it came about

The model follows the Qwen2 series, building on feedback received from developers over three months. Alibaba Cloud focused on increasing knowledge retention and improving reasoning capabilities during this period. The Qwen2.5 series represents a significant expansion, with pre-training data scaled up from 7 trillion to 18 trillion tokens. This release includes various sizes, but the 72 billion parameter version serves as the flagship open-weight model. The technical report was published in December 2024, detailing improvements in both pre-training and post-training stages.

Inside the model

This model uses a transformer architecture with RoPE, SwiGLU, RMSNorm, and Attention QKV bias. It is a dense model, meaning all parameters are active for every token, unlike mixture of experts designs. Training involved 18 trillion tokens of high-quality data. Post-training included supervised fine-tuning with over one million samples and multistage reinforcement learning. These steps improved instruction following and alignment with human preferences. The model employs YaRN techniques to handle long contexts beyond its base configuration of 32,768 tokens.

What it can do

Qwen2.5-72B demonstrates strong performance in coding benchmarks such as HumanEval and mathematics tests like MATH. It outperforms several open and proprietary models in language understanding and reasoning tasks. The model shows competitive results against much larger systems, including Llama-3-405B-Instruct. It excels at generating structured outputs like JSON and understanding tables. Users report improved resilience to diverse system prompts, which helps in role-play scenarios and complex condition setting for chatbots.

How to use it

The model weights are available openly under the Apache 2.0 licence, except for certain variants. Developers can download the base and instruction-tuned versions from platforms like Hugging Face. Quantized versions are also provided for easier deployment. For those preferring hosted solutions, Alibaba Cloud offers API access through Model Studio. The code integrates with the latest Hugging Face transformers library, and deployment using vLLM is recommended for handling long texts efficiently.

Why it matters

The release has been described as one of the largest open-source launches in history. It serves as a foundation for specialized models like Qwen2.5-Coder and Qwen2.5-Math. Many developers have built new applications on top of the Qwen2.5 base models. Its performance challenges larger proprietary systems, offering a cost-effective alternative for enterprises. The model supports the creation of multimodal systems and has influenced subsequent developments in the open-weight community.

Questions people ask about Qwen2.5-72B

What is the parameter count of Qwen2.5-72B?

The model has 72 billion parameters. Specifically, the non-embedding parameter count is listed as 70.0 billion. It is a dense model, so all these parameters are used during inference.

Can I use Qwen2.5-72B for commercial projects?

Yes, the model is generally licensed under Apache 2.0. This permissive licence allows for commercial use, modification, and distribution. You should check the specific licence file in the repository for any exceptions.

How long of a text can Qwen2.5-72B process?

It supports a context window of up to 128,000 tokens. For inputs longer than 32,768 tokens, it uses YaRN techniques to maintain performance. It can generate up to 8,192 tokens in a single response.

Which languages does Qwen2.5-72B support?

It supports over 29 languages. Key languages include English, Chinese, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, and Arabic.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Qwen2.5-72B yet, so the facts on this page come from the sources listed above: huggingface.co/Qwen/Qwen2.5-72B-Instruct, arxiv.org/abs/2412.15115, qwenlm.github.io/blog/qwen2.5/. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.