News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
OLMo 2

OLMo 2

Open language model family with full training transparency

Allen Institute for AI released OLMo 2 in late 2024 as a fully open series of dense autoregressive models. It offers 7 billion, 13 billion and 32 billion parameter variants with complete access to weights, code and training data.

Share on WhatsApp

  • Released with full training data and code
  • Uses dense autoregressive transformer architecture
  • Licensed under Apache 2.0 for open use
  • Includes 7B, 13B and 32B parameter sizes
  • Trained on Dolma and Dolmino Mix 1124

What it is

OLMo 2 is a family of large language models developed by the Allen Institute for AI. Released in November 2024, it includes dense autoregressive transformers with 7 billion, 13 billion and 32 billion parameters. The project aims to support scientific study of language models by providing full transparency. Users can access model weights, training code, logs and the complete dataset used for pretraining. The models are designed for general purpose text generation and instruction following tasks.

How it came about

The OLMo series continues the work of its predecessor by expanding openness in model development. The team published details in a paper titled OLMo 2 Furious in December 2024. This release introduced significant changes to the training recipe and architecture. An update in January 2025 corrected post-trained models that initially lacked the correct pre-tokenization logic. The legacy versions were renamed with a preview suffix while new corrected models replaced them under the original names.

Inside the model

These models use a dense decoder-only transformer architecture. Training relies on the Dolma dataset and a specialized mixture called Dolmino Mix 1124. This mix is applied during the late annealing phase to improve efficiency and stability. The instruct variants undergo supervised fine-tuning using the Tülu 3 dataset. Further alignment uses direct preference optimization and reinforcement learning with verifiable rewards. This process targets performance on mathematical and logical reasoning benchmarks.

What it can do

The 13 billion parameter instruct model achieves an average score of 62.0 across several benchmarks. It scores 87.4 on GSM8k for mathematical problems and 82.6 on IFEval for instruction following. Performance on MMLU reaches 68.5. The models compete with other open-weight systems of similar size. They also show competitive results against some proprietary models like GPT-3.5 Turbo. Safety evaluations indicate limited specific training for harm reduction.

How to use it

All artifacts are available under the Apache 2.0 licence. This includes weights, code, recipes and training logs. The models are hosted on Hugging Face and supported by the Transformers library. Users can load them via standard python scripts. The licence permits research and educational use. Some fine-tuning data includes outputs from third party models subject to additional terms such as the Gemma Terms of Use.

Safety and testing

The developers note that safety training is limited compared to commercial assistants. The models do not include automatic in-the-loop filtering of responses. This means they may produce problematic outputs if prompted to do so. Users are directed to responsible use guidelines for best practices. The open nature allows independent auditing but requires careful deployment by users.

Why it matters

OLMo 2 sets a precedent for full transparency in large model training. By releasing every checkpoint and log, it enables detailed analysis of training dynamics. This approach supports the broader scientific community in understanding how scale and data affect performance. It challenges the trend of closed proprietary systems by proving competitive results with open methods.

Questions people ask about OLMo 2

What makes OLMo 2 different from other open models?

It provides complete transparency by releasing all training data, code, logs and intermediate checkpoints. This allows researchers to study the entire training process rather than just the final weights.

Can I use OLMo 2 for commercial applications?

The model is licensed under Apache 2.0 which generally permits commercial use. However, some fine-tuning data includes third-party outputs subject to additional terms like the Gemma Terms of Use.

How large are the OLMo 2 models?

The family includes three sizes: 7 billion, 13 billion and 32 billion parameters. All are dense autoregressive transformers designed for text generation and instruction following.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on OLMo 2 yet, so the facts on this page come from the sources listed above: huggingface.co/allenai/OLMo-2-1124-13B-Instruct, arxiv.org/abs/2501.00656. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.