News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
Pragna-1B

Pragna-1B

Small decoder model for Hindi, Bangla, Gujarati and English

Pragna-1B is a small language model built by Soket AI Labs. It supports Hindi, Bangla, Gujarati and English. The design draws on TinyLlama and uses efficient attention methods. It aims to run on devices with limited compute power.

Share on WhatsApp

  • Built by Soket AI Labs for Indian languages and English
  • Uses Grouped Query Attention for faster inference
  • Trained on Bhasha-wiki, SlimPajama and Sangraha-Verified
  • Inspired by TinyLlama architecture
  • Supports Hindi, Bangla, Gujarati and English text

What it is

Pragna-1B is a decoder-only transformer model created by Soket AI Labs. It contains roughly one billion parameters. The system handles text in Hindi, Bangla, Gujarati and English. Developers designed it to work efficiently on hardware with lower computational capacity. It serves as a base for tasks requiring regional Indian language support alongside English. The model is available for download and local use.

How it came about

Soket AI Labs developed the model using their GenAI Studio platform. This modular system supports various generative AI architectures. The team trained the model to scale across thousands of graphics processing units. They used Triton, an open-source language from OpenAI, to write custom code for faster operations. The project leverages Fully Sharded Data Parallel methods for distributed training.

Inside the model

The architecture follows a decoder-only transformer pattern inspired by TinyLlama. It uses Rotary Positional Encoding with a base of 10,000 to manage position data. Normalisation relies on RSNorm with a small epsilon value. The activation function is SiLU, or Sigmoid Linear Unit. A key feature is Grouped Query Attention, which reduces memory bandwidth needs compared to standard multi-head attention. Training acceleration comes from FlashAttention2. The dataset mix includes Bhasha-wiki, SlimPajama and Sangraha-Verified collections.

What it can do

Benchmark results show varied performance across languages. In Hindi tests, it scores an average of 0.30 on Arc and Hellaswag tasks. Gujarati evaluation yields a similar average of 0.30. English benchmarks result in an average score of 0.34, with zero on GSM8K math problems. It trails behind larger models like Llama-3-8B and Gemma-7B in these tests. Sample outputs demonstrate text generation in all four supported languages, though factual accuracy in historical claims varies.

How to use it

The model weights are hosted on Hugging Face under the soketlabs account. Users can load it via the Transformers library using bfloat16 precision. Soket AI Labs provides the code for initialisation. The licence allows for open access to the weights, enabling local deployment and further fine-tuning by the community.

Questions people ask about Pragna-1B

What languages does Pragna-1B support?

The model handles text in four languages: Hindi, Bangla, Gujarati and English. It is designed to serve users in India and surrounding regions who need multilingual support.

How large is the Pragna-1B model?

Pragna-1B has approximately one billion parameters. This small size allows it to run on devices with lower computational power compared to larger industry models.

What makes its architecture efficient?

It uses Grouped Query Attention instead of standard Multi-Head Attention. This change reduces memory bandwidth usage and speeds up both training and inference processes.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Pragna-1B yet, so the facts on this page come from the sources listed above: huggingface.co/soketlabs/pragna-1b. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.