ERNIE-4.5-21B-A3B-Thinking
Lightweight reasoning model with 21 billion parameters
Baidu released ERNIE-4.5-21B-A3B-Thinking in 2025 as a lightweight mixture of experts model. It features 21 billion total parameters with 3 billion active per token, designed for complex logical and mathematical reasoning tasks.
- 21 billion total parameters with 3 billion active
- Supports 128K long-context understanding
- Released under Apache License 2.0
- Optimized for complex reasoning tasks
- Compatible with vLLM and FastDeploy
What it is
ERNIE-4.5-21B-A3B-Thinking is a language model developed by Baidu. It belongs to the ERNIE 4.5 family and was released in 2025. The model contains 21 billion parameters in total but activates only 3 billion for each token processed. This design makes it a lightweight option suitable for complex reasoning tasks. It supports a context window of 128,000 tokens and includes capabilities for using external tools.
How it came about
Baidu developed this version over a three-month period to scale the thinking capability of the base ERNIE-4.5-21B-A3B model. The company aimed to improve the quality and depth of reasoning to enhance competitiveness in lightweight model categories. The release focuses on advancing performance in tasks that typically require human expertise.
Inside the model
The architecture uses a mixture of experts approach with 64 text experts in total. For any given token, only 6 experts are activated. This structure allows for efficient processing while maintaining a large parameter count. The model was post-trained and releases Transformer-style weights. It is compatible with PyTorch and PaddlePaddle ecosystems, including tools like vLLM and FastDeploy.
What it can do
The model shows improved performance in logical reasoning, mathematics, science, and coding. It handles text generation and academic benchmarks that demand high-level expertise. Efficient tool usage is a key feature, allowing the model to interact with external functions. The extended context window supports understanding of long documents up to 128K tokens.
How to use it
Weights are available under the Apache License 2.0, which permits commercial use. Developers can load the model using standard libraries such as transformers version 4.54.0 or newer. Inference can be run via FastDeploy version 2.2 or vLLM version 0.10.2 and above. Deployment requires significant GPU resources, specifically 80GB for a single GPU setup.
Questions people ask about ERNIE-4.5-21B-A3B-Thinking
How many parameters does the model have?
The model has 21 billion total parameters. However, it uses a mixture of experts architecture where only 3 billion parameters are activated for each token.
What license covers this model?
Baidu provides the model under the Apache License 2.0. This license allows for commercial use subject to the standard terms and conditions of the agreement.
Can it use external tools?
Yes, the model supports function calls and efficient tool usage. It includes specific parsers for reasoning and tool calls when used with compatible inference engines like vLLM.
Good reading and sources
There is no English Wikipedia article on ERNIE-4.5-21B-A3B-Thinking yet, so the facts on this page come from the sources listed above: huggingface.co/baidu/ERNIE-4.5-21B-A3B-Thinking. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.