News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
OLMo 3

OLMo 3

Open language models for scientific research by Allen Institute for AI

Allen Institute for AI released OLMo 3, a family of 7 billion and 32 billion parameter language models. Trained on the Dolma 3 dataset with full code and checkpoints available under an Apache 2.0 licence, these models aim to support open science in artificial intelligence.

Share on WhatsApp

  • Released under Apache 2.0 licence with full open weights
  • Trained on 5.5 to 5.9 trillion tokens from Dolma 3 dataset
  • Supports context length of 65,536 tokens
  • Includes Base, Instruct, and Think model variants
  • Developed by Allen Institute for AI for open science

What it is

OLMo 3 is a suite of open language models developed by the Allen Institute for AI. It includes two primary sizes, one with 7 billion parameters and another with 32 billion. The project provides Base, Instruct, and Think variants to serve different research and application needs. These models are designed to facilitate the scientific study of language technologies by making all training details, code, and checkpoints publicly accessible. They operate as autoregressive transformers trained primarily on English text data.

How it came about

The OLMo series continues the work of its predecessor, OLMo 2. This latest iteration introduces a staged training approach for its base models. The release includes specific checkpoints dated October and November 2025, such as the Olmo-3-1025-7B and Olmo-3-1125-32B. The development team at Allen Institute for AI has maintained a consistent focus on transparency, releasing this family to enable broader scientific inquiry into how large language models function and improve over time.

Inside the model

These models use a standard decoder-only transformer architecture. The 7 billion parameter version has 32 layers and a hidden size of 4096, while the 32 billion parameter model features 64 layers and a hidden size of 5120. Both support a context window of 65,536 tokens. Training utilized the Dolma 3 dataset, comprising approximately 5.5 to 5.9 trillion tokens depending on the model size. The training process involves multiple stages, including supervised fine-tuning and direct preference optimisation, culminating in reinforcement learning with verified rewards for the Think variants.

What it can do

Evaluation results show performance across various benchmarks including coding, mathematics, and general knowledge. The 32 billion parameter model scores 64.7 on Olmo 3-Eval Math and 48.1 on BigCodeBench. It achieves 65.6 on HumanEval for code generation. In general knowledge tasks, it reaches 79.7 on MMLU STEM and 85.0 on MMLU Humanities. The models demonstrate competitive results against other open-weight systems like Qwen-2.5-32B and Gemma-3-27B in several categories, particularly in reasoning and code-related evaluations.

How to use it

All code, model checkpoints, and training details are released under the Apache 2.0 licence. Users can access the models via the Hugging Face transformers library, requiring version 4.57.0 or higher. The weights are fully open, allowing for local inference, fine-tuning, and integration into custom applications. Quantisation options are available for faster performance, such as loading in 8-bit precision. Intermediate checkpoints from various training stages are also provided for research purposes.

Questions people ask about OLMo 3

What is the licence for OLMo 3?

The models, code, and associated training details are released under the Apache 2.0 licence. This permissive licence allows for broad use, modification, and distribution of the software and model weights.

How large is the context window?

Both the 7 billion and 32 billion parameter versions of OLMo 3 support a context length of 65,536 tokens. This allows the models to process and generate responses based on substantial amounts of input text.

What dataset was used for training?

OLMo 3 models were trained on the Dolma 3 dataset. The 7 billion parameter model processed 5.93 trillion tokens, while the 32 billion parameter model was trained on 5.50 trillion tokens.

Can I fine-tune these models?

Yes, fine-tuning is supported using the OLMo-core repository. Users can start from the final checkpoint or many intermediate checkpoints. Recipes and scripts are provided to override configuration options like learning rate.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on OLMo 3 yet, so the facts on this page come from the sources listed above: huggingface.co/allenai/Olmo-3-1125-32B. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.