ERNIE 3.0
Knowledge-enhanced pre-training model for language tasks
ERNIE 3.0 is a large-scale language model developed by researchers at Baidu, announced in July 2021. It integrates plain text with knowledge graphs to improve performance in both understanding and generation tasks across Chinese and English benchmarks.
- Contains 10 billion parameters
- Trained on 4TB of text and knowledge graphs
- Top score on SuperGLUE in July 2021
- Surpassed human performance on SuperGLUE
- Fuses auto-regressive and auto-encoding networks
What it is
ERNIE 3.0 is a pre-trained language model designed to handle both natural language understanding and generation. Developed by a team at Baidu, it was introduced in mid-2021 with ten billion parameters. The model distinguishes itself by incorporating structured world knowledge alongside standard text data during training. This approach aims to address limitations in earlier large models that relied solely on plain text corpora.
How it came about
The model emerged from research into scaling up pre-trained systems while improving their generalization capabilities. Announced in July 2021 via an arXiv paper, the project sought to move beyond traditional auto-regressive training methods. The developers focused on integrating linguistic and world knowledge directly into the pre-training phase to enhance downstream task performance.
Inside the model
The architecture combines auto-regressive and auto-encoding networks into a single unified framework. This hybrid design allows the model to be adapted easily for various tasks using zero-shot, few-shot, or fine-tuning methods. Training involved a four terabyte corpus composed of plain text and a large-scale knowledge graph. This fusion enables the system to leverage structured data relationships rather than just statistical word patterns.
What it can do
The model demonstrated strong results across fifty-four Chinese natural language processing tasks. In English evaluations, it achieved top placement on the SuperGLUE benchmark as of July 2021. The English version reportedly surpassed human performance by zero point eight percent, scoring ninety point six percent compared to the human baseline of eighty nine point eight percent.
How to use it
The model was presented primarily through academic channels and research publications. The primary documentation available is the technical paper describing its architecture and empirical results. There is no indication in the provided material of a public API or open-weight release for general commercial use at the time of announcement.
Questions people ask about ERNIE 3.0
What makes ERNIE 3.0 different from other large models?
It integrates a large-scale knowledge graph with plain text during pre-training. This allows the model to use structured world knowledge and linguistic data, rather than relying only on statistical patterns found in raw text corpora.
How large is the ERNIE 3.0 model?
The model contains ten billion parameters. It was trained on a dataset totaling four terabytes, which includes both standard plain text and structured information from knowledge graphs.
How did it perform on English benchmarks?
Its English version achieved first place on the SuperGLUE benchmark in July 2021. It scored ninety point six percent, which was zero point eight percent higher than the reported human performance level.
Good reading and sources
There is no English Wikipedia article on ERNIE 3.0 yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2107.02137. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.