Llama
Meta AI's family of open-weight language models, from 1B to 2T parameters
Llama is a family of large language models first published by Meta AI in February 2023 and extended through April 2025. Sizes run from 1 billion to 2 trillion parameters, and the line moved from research-only weights to broadly downloadable releases that powered Meta's assistant and many outside projects.
- First announced 24 February 2023; Llama 4 shipped 5 April 2025
- Family sizes range from 1 billion to 2 trillion parameters
- Llama 4 uses a mixture of experts with 17B active parameters
- Llama 1 weights leaked via BitTorrent in March 2023
- Llama 3 trained on roughly 15 trillion tokens
What it is
Llama is a line of large language models published by Meta AI, beginning in February 2023 and ending with Llama 4 in April 2025. The name is treated as a backronym for Large Language Model Meta AI. Sizes across the family span from 1 billion to 2 trillion parameters. Early releases were plain foundation models, but from Llama 2 onward instruction-tuned chat variants shipped alongside the base weights. The models are used for general text work, coding, and, in the newest generation, image understanding. Meta also built its own assistant, Meta AI, on top of the line.
How it came about
The first Llama arrived on 24 February 2023 through a blog post and a paper covering training, design and results. Weights were handed out through an application process to academics, government and civil-society affiliates and industry labs, while the inference code was published under GPLv3. A torrent of the weights appeared on 4chan on 3 March 2023 and spread widely; Meta issued takedown notices to HuggingFace and GitHub. Llama 2 followed on 18 July 2023 with Microsoft as a partner, then Llama 3 on 18 April 2024, Llama 3.1 on 23 July 2024 and Llama 4 on 5 April 2025.
Inside the model
The series follows the GPT-3 pattern of an autoregressive decoder-only transformer, with SwiGLU in place of GeLU, rotary positional embeddings instead of absolute ones, and RMSNorm rather than layer normalisation. The team pushed performance by adding training data rather than parameters, since running a finished model costs more than building it. Llama 1 saw 1.4 trillion tokens from CommonCrawl pages, GitHub code, Wikipedia in 20 languages, Project Gutenberg, Books3, ArXiv sources and Stack Exchange. Llama 2 used 2 trillion tokens with personal-data sites filtered out. Llama 3 drew on about 15 trillion tokens, mostly English. Llama 4 switched to a mixture of experts.
What it can do
Meta AI reported that the 13B Llama 1 model beat the far larger GPT-3 on most NLP benchmarks, and that the 65B version matched PaLM and Chinchilla. For Llama 3, the company said the 70B model outperformed Gemini Pro 1.5 and Claude 3 Sonnet on most tests in April 2024. Llama 3 also showed that gains continue log-linearly well past the Chinchilla-optimal data size. Llama 4 was claimed to exceed GPT-4o on LMArena, but that score came from an experimental chat build rather than the public release.
How to use it
Llama 1 weights were restricted to approved researchers under a non-commercial licence, with only the inference code openly published. Later versions were released with weights and licences allowing many commercial uses, though an acceptable use policy still bars some purposes. The Open Source Initiative and others dispute Meta's use of the term open source for these releases. Code and model files are hosted in the meta-llama/llama-models repository, and the family has its own website at llama.com.
Safety and testing
Chat variants were aligned with reinforcement learning from human feedback. Annotators wrote prompts, compared pairs of outputs and attached safety labels that could veto a choice, feeding two reward models for safety and helpfulness. Training mixed rejection sampling with proximal policy optimization. A Ghost attention technique kept system instructions consistent across a conversation. Meta filed takedown requests in March 2023 over unauthorised copies of the first model, and HuggingFace and GitHub complied.
Reception and debate
Commentators split over the March 2023 leak. Some warned of misuse such as better spam, while others welcomed cheap local runs of the smaller sizes and compared the moment to Stable Diffusion, whose open distribution had spurred a wave of tools. The Llama 4 launch drew criticism after LMArena said Meta's reading of its rules did not match what it expects from providers, and users on social media accused Meta of tuning a separate model for the leaderboard and of training on test sets, a claim the company denied.
Why it matters
Llama 2 opened the weights to commercial use and became a base for many downstream projects. Stanford's Alpaca applied the Self-Instruct recipe to Llama 7B to reach capabilities near GPT-3 text-davinci-003 at modest cost, before the files were pulled over hosting and safety worries. Together AI's RedPajama effort rebuilt and shared an open version of the Llama 1 dataset, starting near 1.2 trillion tokens. Llama 3 arrived with Meta AI, an assistant on Facebook and WhatsApp.
Questions people ask about Llama
What does the name Llama stand for?
It is treated as a backronym for Large Language Model Meta AI. Earlier write-ups styled it LLaMA or LLaMa. The models were published by Meta AI, part of Meta Platforms, starting in February 2023.
Can I download and use the models commercially?
Not the first one. Llama 1 weights went only to approved researchers under a non-commercial licence. From Llama 2 onward the weights were released with licences permitting many commercial uses, though an acceptable use policy still restricts certain purposes.
How big do these models get?
The family covers 1 billion to 2 trillion parameters. Llama 3.1 topped out at 405B. In Llama 4, Scout totals 109B and Maverick 400B, each using 17B active parameters per token, while the announced Behemoth was described as roughly 2T total.
What is different about Llama 4?
It uses a mixture of experts, so only part of the network fires for each token. It accepts text and images, returns text, and covers 12 languages. Scout offers a 10M token context window and Maverick 1M. Both were released as base and instruction-tuned versions.
Why was the Llama 4 benchmark result disputed?
Meta said Llama 4 beat GPT-4o on LMArena, but the score came from an unreleased experimental chat build tuned for conversation, not the public model. LMArena said it would tighten its policies and that Meta should have flagged the custom version more clearly.
Good reading and sources
- Official websiteOfficial site
- Meta’s powerful AI language model has leaked online — what happens now?The Verge, Article
- LLaMA: Open and Efficient Foundation Language ModelsarXiv, Article
- GGUF · Hugging FaceHugging Face, Article
The facts on this page come from the Wikipedia article "Llama (language model)", written by its contributors and used under the CC BY-SA 4.0 licence, and from its Wikidata record (CC0). EveryLLM rewrote the text in its own words, with AI assistance, and compares every page with the article to make sure no passage is copied. It may be reused under the same CC BY-SA 4.0 licence with a link back.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.