News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
fnLlama 3.1 405B

Llama 3.1 405B

Meta's largest openly available language model, released in 2024

Llama 3.1 405B is a 405-billion-parameter language model released by Meta in July 2024 as the flagship of the Llama 3.1 family. Meta describes it as the first openly available model to match leading closed models on general knowledge, maths, tool use and translation, and it is downloadable for further training and customisation.

Share on WhatsApp

  • 405 billion parameters, Meta's largest Llama model at release
  • Trained on over 15 trillion tokens
  • Training used more than 16,000 H100 GPUs
  • 128K token context window, with support for eight languages
  • Quantized to 8-bit FP8 so it fits in one server node

What it is

Llama 3.1 405B is a large language model with 405 billion parameters, built by Meta and released in July 2024. It is the biggest member of the Llama 3.1 family, which also includes updated 8B and 70B versions. Meta positions it as an openly available model that can stand alongside leading closed systems on tasks such as general knowledge, maths, steerability, tool use and multilingual translation. Because the weights can be downloaded, developers can fine-tune it on their own data, generate synthetic training data with it, or use it to distil smaller models.

How it came about

Meta released Llama 3.1 405B alongside upgraded 8B and 70B models, describing the 405B as its largest model to date. The company says all Llama versions together had passed 300 million downloads by that point. Training the flagship was a major engineering effort: the team optimised the full training stack and pushed the run to more than 16,000 H100 GPUs, the first Llama model trained at that scale. The technical report, 'The Llama 3 Herd of Models', was posted to arXiv on 31 July 2024 and later revised.

Inside the model

The model uses a standard decoder-only transformer with small adaptations. Meta says it deliberately avoided a mixture-of-experts design to keep training stable. It was pretrained on more than 15 trillion tokens, with improved curation and filtering pipelines compared with earlier Llama versions. Post-training ran iteratively: each round combined supervised fine-tuning, rejection sampling and direct preference optimization, using mostly synthetic data that was filtered for quality. For serving, weights were quantized from 16-bit BF16 to 8-bit FP8 so the model can run within a single server node.

What it can do

Meta evaluated the release on more than 150 benchmark datasets across many languages, plus human comparisons against competitors in real-world scenarios. The company reports that the flagship is competitive with GPT-4, GPT-4o and Claude 3.5 Sonnet across a range of tasks, and that the smaller 8B and 70B models hold up against closed and open models of similar size. Supported uses include long-form summarisation, multilingual conversation and coding assistance. The 405B model was also used to improve the post-training quality of the smaller models.

How to use it

Weights are downloadable from llama.meta.com and Hugging Face, and the models run on partner platforms including AWS, NVIDIA, Databricks, Groq, Dell, Azure, Google Cloud and Snowflake. Meta updated its licence to let developers use Llama outputs, including those from the 405B, to improve other models. In the US, the model can be tried on WhatsApp and at meta.ai.

Safety and testing

Meta says it balanced helpfulness with safety during alignment, aiming to keep answers useful as mitigations were added. The release includes a reference system with open-source sample applications, plus new components: Llama Guard 3, a multilingual safety model, and Prompt Guard, a filter for prompt injection attacks.

Why it matters

Meta presents the 405B as a step that brings open models level with closed ones, and expects it to encourage workflows that were hard at this scale before, such as synthetic data generation and distillation of smaller models. The company also began work on 'Llama Stack', a proposed set of standard interfaces for tooling and agentic applications, released as a request for comment on GitHub to improve interoperability across the ecosystem.

Questions people ask about Llama 3.1 405B

What makes Llama 3.1 405B different from earlier Llama models?

It is much larger, extends the context window to 128K tokens, supports eight languages and adds stronger tool use. Meta also changed its licence so outputs from the model, including the 405B, may be used to improve other models.

How can I get the model?

The weights are downloadable from llama.meta.com and Hugging Face, and the model is offered on partner platforms such as AWS, Azure, Google Cloud, NVIDIA, Groq, Dell, Databricks and Snowflake. In the US it can also be tried on WhatsApp and meta.ai.

Why did Meta choose a plain transformer instead of a mixture of experts?

Meta says it picked a standard decoder-only transformer with minor adaptations rather than a mixture-of-experts design in order to maximise training stability at this scale.

How does it perform against other models?

Meta reports results across more than 150 benchmark datasets and human evaluations, saying the flagship is competitive with GPT-4, GPT-4o and Claude 3.5 Sonnet on tasks including general knowledge, maths, tool use and translation.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Llama 3.1 405B yet, so the facts on this page come from the sources listed above: ai.meta.com/blog/meta-llama-3-1/, arxiv.org/abs/2407.21783. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.