News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
Sarvam-105B

Sarvam-105B

Open-weight MoE model for Indian languages and reasoning

Sarvam-105B is a large language model built by Sarvam AI. It uses a mixture of experts design to handle complex reasoning tasks efficiently. The model supports twenty-two Indian languages and offers open weights under an Apache licence.

Share on WhatsApp

  • 105 billion total parameters with 10.3 billion active
  • Supports 22 Indian languages with state-of-the-art performance
  • Open-sourced under the Apache License
  • Scores 98.6 on Math500 benchmark
  • Uses 128 experts with top-8 routing

What it is

Sarvam-105B is a large language model developed by Sarvam AI. It contains 105 billion parameters in total but activates only 10.3 billion for each token processed. This design helps it manage complex tasks like coding and mathematics while remaining efficient. The model focuses heavily on the Indian context and supports twenty-two regional languages. It is released with open weights under the Apache Licence, allowing developers to download and run it locally.

How it came about

The model was introduced by the Sarvam Foundation Models Team as part of their sovereign model initiative. It was launched alongside a smaller thirty-billion parameter variant. The team published technical details and benchmark results on their blog and Hugging Face repository. They also provided code examples for running the model using standard libraries.

Inside the model

The architecture uses a mixture of experts approach with one hundred and twenty-eight specialists. For any given input, the system routes data to the top eight experts plus one shared expert. It employs an MLA-style attention stack with decoupled query-key head dimensions. This structure allows for higher representational bandwidth while keeping the hidden size at 4096. The model supports a context window of 128,000 tokens using YaRN scaling. Training included auxiliary-loss-free router balancing to stabilise performance.

What it can do

The model performs strongly in agentic tasks, web search, and technical troubleshooting. Benchmark scores show high accuracy in mathematics, reaching 98.6 on Math500. It also achieves 71.7 on Live Code Bench v6 and 90.6 on MMLU. In reasoning tests like GPQA Diamond, it scores 78.7. The company claims it matches or surpasses several major closed-source models in these areas. It demonstrates particular strength in handling complex logical problems and coding challenges.

How to use it

Developers can access the model weights via Hugging Face. The release uses the Apache Licence, which permits broad usage. Inference code is provided for transformers, SGLang, and vLLM. Users may need to install specific forks or apply hot patches for vLLM support due to pending pull requests. The model runs on CUDA-enabled GPUs and requires trust remote code settings for initial loading.

Questions people ask about Sarvam-105B

How many parameters does Sarvam-105B have?

The model has 105 billion total parameters. However, it uses a mixture of experts design, so only 10.3 billion parameters are active for each token processed. This makes it more efficient than dense models of similar total size.

Which languages does Sarvam-105B support?

It supports English and twenty-two Indian languages. The training process placed a major focus on the Indian context, resulting in strong performance across these regional languages compared to other models of its size.

Can I use Sarvam-105B commercially?

Yes, the model is open-sourced under the Apache License. This permissive licence allows for commercial use, modification, and distribution. You can download the weights from Hugging Face and integrate them into your own applications.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Sarvam-105B yet, so the facts on this page come from the sources listed above: huggingface.co/sarvamai/sarvam-105b. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.