ERNIE 5.5 Flash
Efficient sparse MoE language model with 280B total parameters
Baidu released ERNIE 5.5 Flash as a base language model trained on 33.5 trillion tokens. It uses a sparse mixture of experts architecture to activate only 14 billion parameters per token while maintaining 280 billion total parameters. The model supports context windows up to one million tokens during inference.
- 280 billion total parameters with 14 billion activated per token
- Trained on 33.5 trillion tokens from web and code sources
- Supports up to 1 million tokens in context during inference
- Uses hybrid attention with 37 compressed and 6 latent layers
- Achieved 93.3 score on 5-shot MMLU benchmark
What it is
ERNIE 5.5 Flash is a large language model developed by Baidu. It functions as a base model, meaning it has undergone pre-training but not yet instruction tuning or alignment for specific chat tasks. The system contains 280 billion parameters in total but activates only 14 billion for each token processed. This design aims to balance high capability with computational efficiency. It handles text inputs and supports context windows of up to one million tokens during inference. The model was trained on 33.5 trillion tokens covering web text, code repositories and long documents.
How it came about
Baidu designed the model to serve as a starting point for further development. The company released the weights at the end of the pre-training phase. This approach allows developers to adapt the model for specific domains or agentic applications. The release includes checkpoints in both FP8 and BF16 precision formats. The model was evaluated alongside other recent base models such as DeepSeek-V4-Flash-Base and Hy3-preview-Base using a unified pipeline.
Inside the model
The architecture uses a sparse mixture of experts structure with 43 layers. One layer is dense while the other 42 use multi-router latent experts. Each expert layer contains 512 routed experts and one shared expert. The system employs hybrid attention mechanisms. Thirty-seven layers use heavily compressed attention to reduce memory costs. Six layers use multi-head latent attention with sparse attention to maintain full-context access. Training utilized the Muon optimizer and a custom tokenizer with approximately 200,000 entries. The data mix prioritized repository-scale code and long-context understanding.
What it can do
The model demonstrates strong performance in knowledge, reasoning, mathematics and code tasks. In five-shot evaluations, it scored 93.3 on MMLU and 94.1 on MMLU-Redux. It achieved 78.4 on MMLU-Pro. These results were compared against similar sized models under identical testing conditions. The system is designed to handle repository-level code reasoning and agentic software engineering tasks. It also supports multilingual capabilities through its diverse training data.
How to use it
Baidu provides the model weights openly via Hugging Face and ModelScope. Developers can download the base checkpoint in FP8 or BF16 precision. The licence permits further training and adaptation. The company encourages users to build upon the general capabilities for domain-specific applications. No API access is mentioned in the provided materials, focusing instead on downloadable weights for research and development.
Questions people ask about ERNIE 5.5 Flash
How many parameters does ERNIE 5.5 Flash have?
The model has 280 billion total parameters. However, it uses a sparse mixture of experts design that activates only 14 billion parameters for each token processed. This allows for efficient computation while maintaining large model capacity.
What is the maximum context length supported?
During pre-training, the model used a context window of up to 128,000 tokens. For fine-tuning and inference, the system supports contexts of up to one million tokens. This enables processing of very long documents and code repositories.
Is the model available for download?
Yes, Baidu has released the base model weights openly. You can download the checkpoints in FP8 or BF16 precision from Hugging Face or ModelScope. The release is intended for developers to build upon for domain adaptation.
Good reading and sources
There is no English Wikipedia article on ERNIE 5.5 Flash yet, so the facts on this page come from the sources listed above: huggingface.co/baidu/ERNIE-5.5-Flash-Base-BF16. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.