News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
ERNIE 5.5 Flash

ERNIE 5.5 Flash

Efficient sparse MoE language model with 280B total parameters

Baidu released ERNIE 5.5 Flash as a base language model trained on 33.5 trillion tokens. It uses a sparse mixture of experts architecture to activate only 14 billion parameters per token while maintaining 280 billion total parameters. The model supports context windows up to one million tokens during inference.

Share on WhatsApp

  • 280 billion total parameters with 14 billion activated per token
  • Trained on 33.5 trillion tokens from web and code sources
  • Supports up to 1 million tokens in context during inference
  • Uses hybrid attention with 37 compressed and 6 latent layers
  • Achieved 93.3 score on 5-shot MMLU benchmark

What it is

ERNIE 5.5 Flash is a large language model developed by Baidu. It functions as a base model, meaning it has undergone pre-training but not yet instruction tuning or alignment for specific chat tasks. The system contains 280 billion parameters in total but activates only 14 billion for each token processed. This design aims to balance high capability with computational efficiency. It handles text inputs and supports context windows of up to one million tokens during inference. The model was trained on 33.5 trillion tokens covering web text, code repositories and long documents.

How it came about

Baidu designed the model to serve as a starting point for further development. The company released the weights at the end of the pre-training phase. This approach allows developers to adapt the model for specific domains or agentic applications. The release includes checkpoints in both FP8 and BF16 precision formats. The model was evaluated alongside other recent base models such as DeepSeek-V4-Flash-Base and Hy3-preview-Base using a unified pipeline.

Inside the model

The architecture uses a sparse mixture of experts structure with 43 layers. One layer is dense while the other 42 use multi-router latent experts. Each expert layer contains 512 routed experts and one shared expert. The system employs hybrid attention mechanisms. Thirty-seven layers use heavily compressed attention to reduce memory costs. Six layers use multi-head latent attention with sparse attention to maintain full-context access. Training utilized the Muon optimizer and a custom tokenizer with approximately 200,000 entries. The data mix prioritized repository-scale code and long-context understanding.

What it can do

The model demonstrates strong performance in knowledge, reasoning, mathematics and code tasks. In five-shot evaluations, it scored 93.3 on MMLU and 94.1 on MMLU-Redux. It achieved 78.4 on MMLU-Pro. These results were compared against similar sized models under identical testing conditions. The system is designed to handle repository-level code reasoning and agentic software engineering tasks. It also supports multilingual capabilities through its diverse training data.

How to use it

Baidu provides the model weights openly via Hugging Face and ModelScope. Developers can download the base checkpoint in FP8 or BF16 precision. The licence permits further training and adaptation. The company encourages users to build upon the general capabilities for domain-specific applications. No API access is mentioned in the provided materials, focusing instead on downloadable weights for research and development.

Questions people ask about ERNIE 5.5 Flash

How many parameters does ERNIE 5.5 Flash have?

The model has 280 billion total parameters. However, it uses a sparse mixture of experts design that activates only 14 billion parameters for each token processed. This allows for efficient computation while maintaining large model capacity.

What is the maximum context length supported?

During pre-training, the model used a context window of up to 128,000 tokens. For fine-tuning and inference, the system supports contexts of up to one million tokens. This enables processing of very long documents and code repositories.

Is the model available for download?

Yes, Baidu has released the base model weights openly. You can download the checkpoints in FP8 or BF16 precision from Hugging Face or ModelScope. The release is intended for developers to build upon for domain adaptation.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on ERNIE 5.5 Flash yet, so the facts on this page come from the sources listed above: huggingface.co/baidu/ERNIE-5.5-Flash-Base-BF16. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.