IndicBERT
Multilingual encoder model for Indian languages
IndicBERT is a multilingual language model designed for natural language understanding in Indian languages. Released in December 2022 by researchers including Sumanth Doddapaneni and Mitesh M. Khapra, it supports 24 languages from four language families. The model aims to improve capabilities for over one billion speakers by leveraging large monolingual corpora.
- Supports 24 languages from 4 language families
- Trained on 20.9 billion tokens from IndicCorp
- Achieves 2-point improvement over baseline benchmarks
- Includes 52 new evaluation sets in IndicXTREME
- Released in December 2022 with open weights
What it is
IndicBERT is a multilingual large language model focused on natural language understanding for Indic languages. It was developed to serve the collective speaker base of more than one billion people across India and surrounding regions. The model supports 24 distinct languages drawn from four different language families. Researchers released it in December 2022 alongside a new benchmark and dataset. Its primary goal is to enhance machine comprehension and processing for these specific linguistic groups.
How it came about
The development team included Sumanth Doddapaneni, Rahul Aralikatte, and Mitesh M. Khapra among others. They submitted their work in December 2022 and revised it in May 2023. The project aimed to address the lack of resources for Indic languages. This effort resulted in the creation of IndicCorp, a large monolingual corpus. The team also introduced IndicXTREME, a human-supervised benchmark for evaluation. IndicBERT v2 was trained to support all the languages covered in these new resources.
Inside the model
The model architecture is an encoder-based transformer designed for multilingual tasks. It was trained on IndicCorp, which contains 20.9 billion tokens. This dataset covers 24 languages and represents a significant increase over prior work. The training process focused on improving zero-shot capabilities, allowing the model to handle tasks without specific examples. The design supports natural language understanding across diverse linguistic structures found in the Indian subcontinent.
What it can do
IndicBERT achieves state-of-the-art performance on the IndicXTREME benchmark. It shows an absolute improvement of two points over strong baselines when averaged across languages and tasks. The model is tested on nine diverse natural language understanding tasks. These evaluations cover 20 languages with a total of 105 evaluation sets. Fifty-two of these sets are new contributions to the literature, testing multilingual zero-shot capabilities.
How to use it
The developers have made both the data and the model weights publicly available. Users can access the resources through the provided URL in the research paper. This open approach allows other researchers and developers to build upon the work. The availability includes the IndicCorp dataset and the IndicBERT v2 model itself. This facilitates further study and application in natural language processing for Indic languages.
Why it matters
This work provides the largest monolingual corpora for Indic languages to date. It establishes a standard benchmark for evaluating multilingual models in this region. By supporting 12 additional languages compared to previous efforts, it broadens the scope of accessible technology. The release encourages further development in natural language understanding for under-resourced languages. It serves as a foundational resource for future models targeting Indian linguistic diversity.
Questions people ask about IndicBERT
How many languages does IndicBERT support?
IndicBERT supports 24 languages from four different language families. This coverage aims to serve the large speaker base of Indic languages, including twelve additional languages not supported by prior major works.
What dataset was used to train IndicBERT?
The model was trained on IndicCorp, a monolingual corpus containing 20.9 billion tokens. This dataset covers 24 languages and represents a 2.3x increase in size compared to previous similar resources.
How does IndicBERT perform on benchmarks?
On the IndicXTREME benchmark, IndicBERT v2 achieves an absolute improvement of two points over strong baselines. This average is calculated across multiple languages and nine diverse natural language understanding tasks.
Good reading and sources
There is no English Wikipedia article on IndicBERT yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2212.05409. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.