YaLM 100B
Open-weight Russian and English language model with 100 billion parameters
Yandex released YaLM 100B, a large language model trained on English and Russian text. It uses 100 billion parameters and was built using DeepSpeed on a cluster of A100 graphics cards. The weights are available under an open licence for research and commercial use.
- Contains 100 billion parameters
- Trained on 300 billion tokens
- Uses 800 A100 GPUs for 65 days
- Licensed under Apache 2.0
- Supports English and Russian text
What it is
YaLM 100B is a pretrained neural network designed to generate and process text. Yandex developed the model, which contains 100 billion parameters. It functions similarly to GPT architectures and supports both English and Russian languages. The company made the model weights freely available to developers and researchers globally. It requires significant hardware resources to run, typically needing multiple graphics processing units with substantial memory.
How it came about
Yandex trained the model on a large cluster of computing hardware. The training process lasted approximately 65 days. During this period, the system processed 300 billion tokens. The team used 800 Nvidia A100 graphics cards to complete the work. Details regarding acceleration and stabilisation techniques were published in technical articles on Medium and Habr. The code provided for inference is based on stock examples from DeepSpeed.
Inside the model
The architecture is a dense decoder transformer inspired by Megatron-LM. Training relied on the DeepSpeed library for efficiency. The dataset comprised 1.7 terabytes of text, split between English and Russian sources. Twenty-five percent of the tokens came from The Pile, an open English dataset. The remaining seventy-five percent consisted of Russian texts, including web pages, news, books, and social media dialogues. Yandex applied filters for length, entropy, and duplication to clean the Russian web data.
What it can do
The model performs text generation and processing tasks. It supports interactive generation, conditional sampling, and greedy decoding strategies. Users can adjust temperature and top-p settings for varied outputs. The system was tested on configurations using four A100 cards or eight V100 cards. It requires about 200 gigabytes of total GPU memory to operate correctly. The model handles few-shot problem solving through its greedy generation mode.
How to use it
Yandex distributes the model under the Apache 2.0 licence. This permits both academic research and commercial applications. Weights and vocabulary files are available for download via GitHub or Hugging Face. The total download size requires roughly 200 gigabytes of disk space. Docker images are also provided for easier deployment on compatible hardware. The code repository includes scripts for running interactive and unconditional generation tasks.
Questions people ask about YaLM 100B
What licence covers YaLM 100B?
The model is released under the Apache 2.0 licence. This allows developers and researchers to use it for both commercial products and academic studies without restrictive conditions.
How much computing power was needed for training?
Training took 65 days on a cluster of 800 Nvidia A100 graphics cards. The model processed 300 billion tokens during this period to learn language patterns.
What languages does the model support?
YaLM 100B supports English and Russian. The training data included twenty-five percent English text from The Pile and seventy-five percent Russian text from various sources.
Can I run this model on a single GPU?
No, the model requires multiple GPUs. It needs approximately 200 gigabytes of total GPU memory. Tests used four A100 cards or eight V100 cards to run inference effectively.
Good reading and sources
There is no English Wikipedia article on YaLM 100B yet, so the facts on this page come from the sources listed above: github.com/yandex/YaLM-100B, huggingface.co/yandex/yalm-100b. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.