Llama 3
Meta's open-weight language model family in 8B and 70B sizes
Llama 3 is a family of open language models released by Meta in 2024, starting with 8B and 70B parameter versions in both pretrained and instruction-tuned forms. It improves on Llama 2 with a larger training set and better reasoning, and powers the Meta AI assistant.
- First release: 8B and 70B parameter models, pretrained and instruction-tuned
- Pretrained on over 15 trillion tokens from public sources
- Tokenizer vocabulary of 128K tokens with grouped query attention
- Human evaluation set of 1,800 prompts across 12 use cases
- Trained across two custom 24K-GPU clusters
What it is
Llama 3 is a set of openly available language models from Meta, put out in 2024 as the next step after Llama 2. The first release covers two sizes, 8B and 70B parameters, each offered as a raw pretrained model and as an instruction-tuned one. Meta says they handle a wide range of jobs, with gains in reasoning, code generation and following instructions. The company frames them as the strongest open models in their size classes and as a base for the Meta AI assistant.
How it came about
Meta built Llama 3 as the successor to Llama 2, aiming to match the best closed models while staying open. The first two sizes arrived as text-only models, with the company promising later versions that would add more languages, images, longer context and extra sizes. A research paper on the family, titled The Llama 3 Herd of Models, was posted in July 2024 and revised that November. Meta says it is releasing early and often so builders can use the models while work continues.
Inside the model
The design is a standard decoder-only transformer, with grouped query attention in both sizes to speed up inference. Text is split using a 128K-token vocabulary, and training runs used 8,192-token sequences with a mask stopping attention from crossing document edges. Pretraining drew on more than 15 trillion tokens from public sources, seven times the Llama 2 set and with four times as much code. Over 5% covers 30-plus non-English languages. Filtering used heuristics, safety screens, deduplication and quality classifiers built with Llama 2. Training combined data, model and pipeline parallelism across two 24K-GPU clusters.
What it can do
Meta reports strong results on standard benchmarks and on a human evaluation set of 1,800 prompts across 12 task types, including advice, coding, creative writing and summarisation. On that set, human raters preferred the 70B instruction-tuned model over Claude Sonnet, Mistral Medium and GPT-3.5 in real-world style tasks. Post-training cut false refusals, improved alignment and widened response variety. The company notes that non-English performance will lag English, and that gains kept rising well past the textbook training budget for a model of this size.
How to use it
Both sizes are released as open weights for broad use, in pretrained and instruction-tuned forms. Meta said the models would reach many hosting platforms and hardware vendors, and the company offers trust and safety tools alongside them. The weights come under a community licence rather than a fully permissive one.
Safety and testing
Meta says it is developing Llama 3 responsibly and ships supporting tools: Llama Guard 2 for content screening, Code Shield for code safety, and CyberSec Eval 2 for security testing. Data pipelines include not-safe-for-work filters and quality checks. The company describes responsible use and deployment of language models as a continuing priority.
Reception and debate
Meta positions Llama 3 as a major step beyond Llama 2 and as the best open model at the 8B and 70B scales, claims made in the company's own announcement. Preference rankings from its private human evaluation set are offered as evidence of real-world strength against named rivals.
Why it matters
Meta says the release is meant to spark the next wave of work across the stack, from apps and developer tools to evaluations and inference tuning. Llama 3 technology backs the Meta AI assistant, which the company describes as a leading assistant for learning, work and creation. Further capabilities, longer context and more sizes were promised, along with a research paper on the family.
Questions people ask about Llama 3
What sizes does Llama 3 come in?
The first release covers two parameter counts, 8 billion and 70 billion. Each is available both as a pretrained base model and as an instruction-tuned version. Meta said more sizes would follow later.
How is it different from Llama 2?
Meta says the training set is seven times larger and holds four times as much code, the tokenizer is far more efficient, and post-training cut false refusals while improving alignment. The company describes gains in reasoning, coding and instruction following.
Can I download and run it?
Yes. The weights are openly released for broad use under a community licence, and Meta said the models would be offered through many platforms and hardware vendors.
Does it handle languages other than English?
Over 5% of the pretraining data covers more than 30 languages, added to prepare for multilingual use. Meta cautions that quality in those languages will not match English.
What safety tools come with it?
Meta ships Llama Guard 2 for screening content, Code Shield for safer code output, and CyberSec Eval 2 for security assessment, alongside filtering used during data preparation.
Good reading and sources
There is no English Wikipedia article on Llama 3 yet, so the facts on this page come from the sources listed above: ai.meta.com/blog/meta-llama-3/, arxiv.org/abs/2407.21783. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.