News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
ERNIE 3.0

ERNIE 3.0

Knowledge-enhanced pre-training model for language tasks

ERNIE 3.0 is a large-scale language model developed by researchers at Baidu, announced in July 2021. It integrates plain text with knowledge graphs to improve performance in both understanding and generation tasks across Chinese and English benchmarks.

Share on WhatsApp

  • Contains 10 billion parameters
  • Trained on 4TB of text and knowledge graphs
  • Top score on SuperGLUE in July 2021
  • Surpassed human performance on SuperGLUE
  • Fuses auto-regressive and auto-encoding networks

What it is

ERNIE 3.0 is a pre-trained language model designed to handle both natural language understanding and generation. Developed by a team at Baidu, it was introduced in mid-2021 with ten billion parameters. The model distinguishes itself by incorporating structured world knowledge alongside standard text data during training. This approach aims to address limitations in earlier large models that relied solely on plain text corpora.

How it came about

The model emerged from research into scaling up pre-trained systems while improving their generalization capabilities. Announced in July 2021 via an arXiv paper, the project sought to move beyond traditional auto-regressive training methods. The developers focused on integrating linguistic and world knowledge directly into the pre-training phase to enhance downstream task performance.

Inside the model

The architecture combines auto-regressive and auto-encoding networks into a single unified framework. This hybrid design allows the model to be adapted easily for various tasks using zero-shot, few-shot, or fine-tuning methods. Training involved a four terabyte corpus composed of plain text and a large-scale knowledge graph. This fusion enables the system to leverage structured data relationships rather than just statistical word patterns.

What it can do

The model demonstrated strong results across fifty-four Chinese natural language processing tasks. In English evaluations, it achieved top placement on the SuperGLUE benchmark as of July 2021. The English version reportedly surpassed human performance by zero point eight percent, scoring ninety point six percent compared to the human baseline of eighty nine point eight percent.

How to use it

The model was presented primarily through academic channels and research publications. The primary documentation available is the technical paper describing its architecture and empirical results. There is no indication in the provided material of a public API or open-weight release for general commercial use at the time of announcement.

Questions people ask about ERNIE 3.0

What makes ERNIE 3.0 different from other large models?

It integrates a large-scale knowledge graph with plain text during pre-training. This allows the model to use structured world knowledge and linguistic data, rather than relying only on statistical patterns found in raw text corpora.

How large is the ERNIE 3.0 model?

The model contains ten billion parameters. It was trained on a dataset totaling four terabytes, which includes both standard plain text and structured information from knowledge graphs.

How did it perform on English benchmarks?

Its English version achieved first place on the SuperGLUE benchmark in July 2021. It scored ninety point six percent, which was zero point eight percent higher than the reported human performance level.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on ERNIE 3.0 yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2107.02137. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.