News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
Tülu 3

Tülu 3

Open post-trained model family built on Llama 3.1

Tülu 3 is a family of instruction-following language models developed by the Allen Institute for AI. Released in late 2024, it builds on Meta’s Llama 3.1 base models using fully open training data and recipes. The project aims to provide transparency in post-training techniques like reinforcement learning with verifiable rewards.

Share on WhatsApp

  • Built on Llama 3.1 base models
  • Uses Reinforcement Learning with Verifiable Rewards
  • Fully open data, code, and training recipes
  • Available in 8B, 70B, and 405B sizes
  • Released by Allen Institute for AI in 2024

What it is

Tülu 3 is a family of large language models designed for instruction following. It was created by the Allen Institute for AI and released in November 2024. The models are built on top of Meta’s Llama 3.1 base architectures, available in sizes of 8 billion, 70 billion, and 405 billion parameters. The project focuses on providing a fully open post-training package, including data, code, and recipes. This approach aims to serve as a guide for modern training techniques while achieving high performance on diverse tasks such as mathematics and coding.

How it came about

The development of Tülu 3 represents a step towards training fully open-source models, similar to the OLMo series. The team introduced the model family in late 2024, publishing a detailed paper on arXiv. This release followed earlier versions in the Tülu line. The researchers aimed to bridge the gap between proprietary and open post-training methods by sharing their complete recipe. The project includes substantial decontamination of existing open datasets to ensure fair benchmarking.

Inside the model

The models use a decoder-only transformer architecture inherited from Llama 3.1. Training involves three main stages: supervised finetuning (SFT), Direct Preference Optimization (DPO), and Reinforcement Learning with Verifiable Rewards (RLVR). RLVR is a novel method introduced in this work. The training data combines public, synthetic, and human-created sources. The team also released a multi-task evaluation scheme and robust toolkit for data curation. Hyperparameters include a linear learning rate schedule and specific PPO settings for the reinforcement learning phase.

What it can do

Tülu 3 models perform strongly on benchmarks like MATH, GSM8K, and IFEval. The 405 billion parameter version achieves scores comparable to or exceeding closed models like GPT-4o-mini and Claude 3.5-Haiku in certain metrics. For instance, the 405B model scores 95.5 on GSM8K and 87.0 on MMLU. The smaller 8B and 70B variants also show improvements over their base Llama 3.1 instruct counterparts in many categories. However, performance varies across different evaluation sets, with some proprietary models still leading in specific areas like AlpacaEval.

How to use it

The model weights, training code, and data are fully open source. Users can download the models from Hugging Face under the Llama 3.1 Community License Agreement. The team provides scripts for loading the model with standard libraries like Transformers and VLLM. A demo is available on the Allen Institute playground. The open nature of the release allows researchers to reproduce the training process and adapt the recipes for other domains.

Safety and testing

The models have limited safety training compared to commercial assistants. They do not include in-the-loop filtering of responses. Consequently, they can produce problematic outputs if prompted to do so. The developers note that the composition of the base Llama 3.1 training corpus is unknown but likely includes web data. Safety benchmarks show varying results, with the 8B model scoring 85.5 on average across six safety tasks.

Why it matters

Tülu 3 serves as a comprehensive guide for open post-training techniques. By releasing the full recipe, including datasets and infrastructure code, it aims to increase transparency in the field. The introduction of RLVR and the multi-task evaluation scheme provides new tools for researchers. The project supports the broader goal of creating fully open-source language models, offering an alternative to proprietary systems.

Questions people ask about Tülu 3

What is Tülu 3?

Tülu 3 is a family of instruction-following language models developed by the Allen Institute for AI. It is built on Meta’s Llama 3.1 base models and is notable for its fully open post-training recipe, including data and code.

How large are the Tülu 3 models?

The Tülu 3 family includes three model sizes: 8 billion parameters, 70 billion parameters, and 405 billion parameters. Each size undergoes the same post-training process involving SFT, DPO, and RLVR.

Is Tülu 3 safe to use?

The models have limited safety training and lack in-the-loop filtering. They can produce harmful or problematic outputs if prompted. Users should be cautious when deploying them without additional safety measures.

What makes Tülu 3 different from other open models?

Tülu 3 releases the complete post-training recipe, including datasets, code, and a novel method called Reinforcement Learning with Verifiable Rewards. This transparency aims to help researchers reproduce and improve upon the results.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Tülu 3 yet, so the facts on this page come from the sources listed above: huggingface.co/allenai/Llama-3.1-Tulu-3-405B, arxiv.org/abs/2411.15124. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.