OLMo 2
Open language model family with full training transparency
Allen Institute for AI released OLMo 2 in late 2024 as a fully open series of dense autoregressive models. It offers 7 billion, 13 billion and 32 billion parameter variants with complete access to weights, code and training data.
- Released with full training data and code
- Uses dense autoregressive transformer architecture
- Licensed under Apache 2.0 for open use
- Includes 7B, 13B and 32B parameter sizes
- Trained on Dolma and Dolmino Mix 1124
What it is
OLMo 2 is a family of large language models developed by the Allen Institute for AI. Released in November 2024, it includes dense autoregressive transformers with 7 billion, 13 billion and 32 billion parameters. The project aims to support scientific study of language models by providing full transparency. Users can access model weights, training code, logs and the complete dataset used for pretraining. The models are designed for general purpose text generation and instruction following tasks.
How it came about
The OLMo series continues the work of its predecessor by expanding openness in model development. The team published details in a paper titled OLMo 2 Furious in December 2024. This release introduced significant changes to the training recipe and architecture. An update in January 2025 corrected post-trained models that initially lacked the correct pre-tokenization logic. The legacy versions were renamed with a preview suffix while new corrected models replaced them under the original names.
Inside the model
These models use a dense decoder-only transformer architecture. Training relies on the Dolma dataset and a specialized mixture called Dolmino Mix 1124. This mix is applied during the late annealing phase to improve efficiency and stability. The instruct variants undergo supervised fine-tuning using the Tülu 3 dataset. Further alignment uses direct preference optimization and reinforcement learning with verifiable rewards. This process targets performance on mathematical and logical reasoning benchmarks.
What it can do
The 13 billion parameter instruct model achieves an average score of 62.0 across several benchmarks. It scores 87.4 on GSM8k for mathematical problems and 82.6 on IFEval for instruction following. Performance on MMLU reaches 68.5. The models compete with other open-weight systems of similar size. They also show competitive results against some proprietary models like GPT-3.5 Turbo. Safety evaluations indicate limited specific training for harm reduction.
How to use it
All artifacts are available under the Apache 2.0 licence. This includes weights, code, recipes and training logs. The models are hosted on Hugging Face and supported by the Transformers library. Users can load them via standard python scripts. The licence permits research and educational use. Some fine-tuning data includes outputs from third party models subject to additional terms such as the Gemma Terms of Use.
Safety and testing
The developers note that safety training is limited compared to commercial assistants. The models do not include automatic in-the-loop filtering of responses. This means they may produce problematic outputs if prompted to do so. Users are directed to responsible use guidelines for best practices. The open nature allows independent auditing but requires careful deployment by users.
Why it matters
OLMo 2 sets a precedent for full transparency in large model training. By releasing every checkpoint and log, it enables detailed analysis of training dynamics. This approach supports the broader scientific community in understanding how scale and data affect performance. It challenges the trend of closed proprietary systems by proving competitive results with open methods.
Questions people ask about OLMo 2
What makes OLMo 2 different from other open models?
It provides complete transparency by releasing all training data, code, logs and intermediate checkpoints. This allows researchers to study the entire training process rather than just the final weights.
Can I use OLMo 2 for commercial applications?
The model is licensed under Apache 2.0 which generally permits commercial use. However, some fine-tuning data includes third-party outputs subject to additional terms like the Gemma Terms of Use.
How large are the OLMo 2 models?
The family includes three sizes: 7 billion, 13 billion and 32 billion parameters. All are dense autoregressive transformers designed for text generation and instruction following.
Good reading and sources
There is no English Wikipedia article on OLMo 2 yet, so the facts on this page come from the sources listed above: huggingface.co/allenai/OLMo-2-1124-13B-Instruct, arxiv.org/abs/2501.00656. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.