Sarvam-105B
Open-weight MoE model for Indian languages and reasoning
Sarvam-105B is a large language model built by Sarvam AI. It uses a mixture of experts design to handle complex reasoning tasks efficiently. The model supports twenty-two Indian languages and offers open weights under an Apache licence.
- 105 billion total parameters with 10.3 billion active
- Supports 22 Indian languages with state-of-the-art performance
- Open-sourced under the Apache License
- Scores 98.6 on Math500 benchmark
- Uses 128 experts with top-8 routing
What it is
Sarvam-105B is a large language model developed by Sarvam AI. It contains 105 billion parameters in total but activates only 10.3 billion for each token processed. This design helps it manage complex tasks like coding and mathematics while remaining efficient. The model focuses heavily on the Indian context and supports twenty-two regional languages. It is released with open weights under the Apache Licence, allowing developers to download and run it locally.
How it came about
The model was introduced by the Sarvam Foundation Models Team as part of their sovereign model initiative. It was launched alongside a smaller thirty-billion parameter variant. The team published technical details and benchmark results on their blog and Hugging Face repository. They also provided code examples for running the model using standard libraries.
Inside the model
The architecture uses a mixture of experts approach with one hundred and twenty-eight specialists. For any given input, the system routes data to the top eight experts plus one shared expert. It employs an MLA-style attention stack with decoupled query-key head dimensions. This structure allows for higher representational bandwidth while keeping the hidden size at 4096. The model supports a context window of 128,000 tokens using YaRN scaling. Training included auxiliary-loss-free router balancing to stabilise performance.
What it can do
The model performs strongly in agentic tasks, web search, and technical troubleshooting. Benchmark scores show high accuracy in mathematics, reaching 98.6 on Math500. It also achieves 71.7 on Live Code Bench v6 and 90.6 on MMLU. In reasoning tests like GPQA Diamond, it scores 78.7. The company claims it matches or surpasses several major closed-source models in these areas. It demonstrates particular strength in handling complex logical problems and coding challenges.
How to use it
Developers can access the model weights via Hugging Face. The release uses the Apache Licence, which permits broad usage. Inference code is provided for transformers, SGLang, and vLLM. Users may need to install specific forks or apply hot patches for vLLM support due to pending pull requests. The model runs on CUDA-enabled GPUs and requires trust remote code settings for initial loading.
Questions people ask about Sarvam-105B
How many parameters does Sarvam-105B have?
The model has 105 billion total parameters. However, it uses a mixture of experts design, so only 10.3 billion parameters are active for each token processed. This makes it more efficient than dense models of similar total size.
Which languages does Sarvam-105B support?
It supports English and twenty-two Indian languages. The training process placed a major focus on the Indian context, resulting in strong performance across these regional languages compared to other models of its size.
Can I use Sarvam-105B commercially?
Yes, the model is open-sourced under the Apache License. This permissive licence allows for commercial use, modification, and distribution. You can download the weights from Hugging Face and integrate them into your own applications.
Good reading and sources
There is no English Wikipedia article on Sarvam-105B yet, so the facts on this page come from the sources listed above: huggingface.co/sarvamai/sarvam-105b. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.