Param-1
Bilingual and multilingual language models from India.
Param-1 is a family of language models developed by BharatGen in India. It includes a 2.9 billion parameter bilingual model and a 7 billion parameter multilingual mixture of experts model. The project aims to build generative AI technologies specifically for the Indian linguistic landscape.
- Includes a 7B parameter Mixture of Experts model.
- Supports English, Hindi, and 14 other Indian languages.
- Trained on up to 7.5 trillion tokens.
- Released under non-commercial or open licenses.
- Developed by BharatGen in India.
What it is
Param-1 is a suite of generative AI models created by BharatGen. The family includes an early checkpoint of a 2.9 billion parameter instruct model and a 7 billion parameter base model. These systems are designed to handle text generation in English, Hindi, and other Indian languages. The project represents a bottom-up initiative to develop large language models within India using limited resources.
How it came about
The development of Param-1 is described as a home-grown effort initiated in India. The team released an early checkpoint of the 2.9 billion parameter model to showcase progress and gather community feedback. This release is intended to encourage research and experimentation. The larger 7 billion parameter model was developed under the BharatGen programme, which aims to provide generative AI technologies for the country.
Inside the model
The 2.9 billion parameter model uses a decoder-only architecture with grouped-query attention and rotary positional embeddings. It was pretrained on 7.5 trillion tokens using 512 H100 graphics processing units. The 7 billion parameter variant employs a mixture of experts design, where only eight of sixty-four experts activate per token. This larger model was trained on four trillion tokens using the NVIDIA NeMo framework on Yotta Shakti Cloud infrastructure. Data curation involved clustering-based iterative techniques to balance linguistic representation.
What it can do
The smaller model supports a context window of 2048 tokens, while the larger model extends this to 4096 tokens. Benchmarks for the 2.9 billion parameter version show zero-shot performance on tasks such as ARC Challenge, HellaSwag, and MMLU in both English and Hindi. The 7 billion parameter model focuses on multilingual understanding across sixteen Indian languages. Both models are pretrained bases that may require further fine-tuning for specific instruction-following tasks.
How to use it
The 2.9 billion parameter instruct checkpoint is released under an open license with responsible usage guidelines. It is intended for research, debugging, and community benchmarking. The 7 billion parameter mixture of experts model is available under the BharatGen non-commercial license. Users must adhere to ethical practices and avoid malicious deployment. The developers encourage the open source community to build India-specific use cases on top of these foundations.
Safety and testing
The early checkpoint of the 2.9 billion parameter model has not undergone full safety alignment or reinforcement learning from human feedback. The developers warn that it may generate biased or unsafe content and is susceptible to prompt injection. It should not be deployed in production environments. The larger model notes that outputs may reflect biases present in web data. Future iterations aim to introduce better alignment and more curated datasets to address these issues.
Why it matters
Param-1 aims to foster innovation and collaboration within the Indian artificial intelligence community. By providing open weights, the project seeks to motivate developers to create localised applications. It represents an effort to contribute meaningfully to the global open-source ecosystem despite resource constraints. The release encourages transparency and dialogue among researchers working on multilingual language technologies.
Questions people ask about Param-1
What languages does Param-1 support?
The models support English and Hindi. The 7 billion parameter version also covers fourteen additional Indian languages, enabling broader linguistic inclusivity across the region.
Is Param-1 free to use?
The 2.9 billion parameter model uses an open license, while the 7 billion parameter model is under a non-commercial license. Users must follow ethical guidelines and credit the authors.
Can I use Param-1 for commercial products?
The 7 billion parameter model is restricted to non-commercial use. The smaller early checkpoint is for research and experimentation. Production deployment is discouraged due to lack of full safety alignment.
What hardware was used to train Param-1?
The 2.9 billion parameter model was pretrained on 512 H100 GPUs. The 7 billion parameter model was trained using the Yotta Shakti Cloud infrastructure with the NVIDIA NeMo framework.
Good reading and sources
There is no English Wikipedia article on Param-1 yet, so the facts on this page come from the sources listed above: huggingface.co/bharatgenai/Param-1-2.9B-Instruct, huggingface.co/bharatgenai/Param-1-7B. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.