Project Indus
Open-source language model for Hindi and dialects
Project Indus is an open-source foundational language model built for Hindi and its dialects. Developed by Makers Lab at TechMahindra, it uses a GPT-2 architecture to support text generation and conversation in Indian languages.
- Trained on 22 billion tokens of Hindi and dialect text
- Built on GPT-2 architecture with custom tokenizer
- Supports dialects like Bhojpuri and Maithili
- Developed by Makers Lab at TechMahindra
- Evaluated on Indic LLM Leaderboard
What it is
Project Indus is a foundational language model designed specifically for Hindi and its associated dialects. It was developed by Nikhil Malhotra, Nilesh Brahme, Satish Mishra, and Vinay Sharma at Makers Lab within TechMahindra. The model aims to enhance natural language processing for Indian linguistic applications. It supports tasks such as question answering and conversation. Built on a GPT-2 architecture, it serves as an open-source resource for researchers and developers focusing on regional language diversity.
How it came about
The model was built from the ground up, starting with a custom tokenizer and extending to the decoder structure. Developers collected data from various sources, including news portals, Wikipedia, and fieldwork for dialects like Maithili and Bhojpuri. The training process utilized high-performance computing resources provided by the Centre for Development of Advanced Computing (CDAC). This infrastructure featured NVIDIA A100 GPUs to handle the extensive preprocessing and supervised learning phases required for the project.
Inside the model
Project Indus uses a decoder-only transformer design based on GPT-2. It was pre-trained on 22 billion tokens derived from approximately 200 GB of cleaned text. The dataset included open-source Hindi content, translated English data from The Pile using IndicTrans2, and digitized books for dialects. A custom Byte Pair Encoding tokenizer with byte fallback was created to handle Hindi script nuances. The team also developed a Bias Removal Toolkit to filter harmful stereotypes from the training data before supervised fine-tuning.
What it can do
The model performs text generation and simulation of conversations in Hindi and dialects such as Bhojpuri, Maithili, and Braj Bhasha. It is not suitable for high-stakes decisions like medical diagnosis or legal advice. Evaluation on the Indic LLM Leaderboard showed an accuracy score of 0.2891 across all tasks. For the ARC-Challenge in Hindi, it achieved an accuracy of 0.2167. The developers note that as a generative model, it may produce hallucinations or unintentional biases despite mitigation efforts.
How to use it
Project Indus is available as an open-source model hosted on Hugging Face. Users can download the weights for local integration and further development. The model has been tested on environments with NVIDIA GeForce RTX 3070 GPUs or higher, as well as Intel Xeon Platinum CPUs. It is intended for direct use in text generation tasks without additional training, though reward tuning is suggested for specific industry applications.
Safety and testing
The developers acknowledge that generative models may produce disturbing stereotypes or harmful content. They employed a Bias Removal Toolkit during data preprocessing to detect and remove biased language. However, they state that any remaining bias is unintentional. The model card recommends regular updates and community feedback to address emergent risks. It explicitly advises against using the model for fill-in-the-blank exercises or multiple-choice questions at this stage.
Questions people ask about Project Indus
What languages does Project Indus support?
The model focuses on Hindi and its dialects, including Bhojpuri, Maithili, Magahi, and Braj Bhasha. It was designed to handle the linguistic diversity of these Indian languages through specialized training data and tokenization.
Is Project Indus free to use?
Yes, it is an open-source foundational model hosted on Hugging Face. Developers and researchers can download the weights and integrate the model into their applications without licensing fees, subject to open-source terms.
What hardware is needed to run it?
The model has been tested on systems with NVIDIA GeForce RTX 3070 GPUs or higher. It can also run on CPU environments, such as those with Intel Xeon Platinum 8580 processors, though performance may vary compared to GPU setups.
Good reading and sources
There is no English Wikipedia article on Project Indus yet, so the facts on this page come from the sources listed above: huggingface.co/nickmalhotra/ProjectIndus. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.