Qwen1.5
Open-weight language model series with nine sizes and strong multilingual support.
Qwen1.5 is a family of decoder-only language models released by the Qwen Team in February 2024. It offers nine dense sizes and one mixture of experts variant, supporting up to 32,768 tokens of context. The series focuses on improved alignment with human preferences and broad multilingual capabilities across twelve languages.
- Supports context length of 32,768 tokens across all sizes.
- Includes nine dense models and one mixture of experts variant.
- Integrated into Hugging Face transformers without remote code trust.
- Outperforms Llama2-70B on multiple standard benchmarks.
- Strong multilingual performance across twelve diverse languages.
What it is
Qwen1.5 is a series of large language models developed by the Qwen Team. Released in February 2024, it serves as a beta version preceding Qwen2. The family includes nine dense models ranging from 0.5 billion to 110 billion parameters, plus a mixture of experts model. These are decoder-only transformers designed for general purpose tasks. They support a context window of 32,768 tokens across all sizes. The release emphasizes better alignment with human instructions and robust performance in multiple languages.
How it came about
The Qwen Team announced Qwen1.5 just before the Chinese New Year in 2024. This update followed earlier versions in the Qwen series. The developers focused on optimizing the experience for software engineers and researchers. They integrated the code directly into Hugging Face transformers library version 4.37.0. This move removed the need for users to trust remote code during installation. The team also collaborated with deployment frameworks like vLLM and SGLang to ensure broad compatibility.
Inside the model
The architecture relies on transformer decoders with SwiGLU activation functions. It uses attention mechanisms with query-key-value biases. Most models employ group query attention, except for the smallest sizes in this beta release. The tokenizer was improved to handle multiple natural languages and programming code effectively. Training involved pretraining on a large dataset. Post-training used supervised fine-tuning and direct preference optimization. Proximal policy optimization was also applied to align chat models with human preferences.
What it can do
Evaluations show strong results in language understanding, coding, and mathematics. The 72 billion parameter model outperforms Llama2-70B on benchmarks like MMLU and C-Eval. Smaller models under 7 billion parameters compete well with other compact models such as Phi-2 and Gemma-2B. Multilingual tests covered twelve languages including Arabic, Japanese, and Thai. The models performed strongly in exams, translation, and reasoning tasks. Chat models scored highly on MT-Bench and Alpaca-Eval, surpassing Claude-2.1 and GPT-3.5-Turbo-0613.
How to use it
Weights are available openly for both base and chat variants. Quantized versions in Int4, Int8, AWQ, and GGUF formats are provided. Developers can access the models via API on DashScope and together.ai. Local inference is supported through Ollama, LMStudio, and llama.cpp. Fine-tuning tools like Axolotl and LLaMA-Factory include support for this series. The integration with standard libraries allows easy loading without special security permissions.
Questions people ask about Qwen1.5
What is the largest model size in Qwen1.5?
The largest dense model in the Qwen1.5 series has 110 billion parameters. There is also a mixture of experts model with 14 billion total parameters but only 2.7 billion active per token.
Does Qwen1.5 support languages other than English?
Yes, it supports twelve languages including Arabic, Spanish, French, German, Japanese, Korean, and Thai. Evaluations show strong performance in translation and understanding across these languages.
How long is the context window for Qwen1.5?
All models in the Qwen1.5 series uniformly support a context length of up to 32,768 tokens. This allows the model to process and generate text based on a significant amount of prior input.
Good reading and sources
There is no English Wikipedia article on Qwen1.5 yet, so the facts on this page come from the sources listed above: qwenlm.github.io/blog/qwen1.5/, huggingface.co/Qwen/Qwen1.5-110B-Chat. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.