News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
Llama Guard

Llama Guard

A Llama 2 based model that classifies prompts and replies for safety risks

Llama Guard is a safety classification model built on Llama2-7b and instruction-tuned on a purpose-built dataset. It was introduced by Meta researchers in a paper posted 7 December 2023, and it checks both user prompts and model responses against a safety risk taxonomy, with weights released for further research.

Share on WhatsApp

  • Built on Llama2-7b, instruction-tuned for safety classification
  • Screens both user prompts and model responses
  • Uses a safety risk taxonomy that can be swapped via prompting
  • Weights released for researchers to adapt
  • Reported to match or beat existing moderation tools on two benchmarks

What it is

Llama Guard is a safeguard model for conversations between people and AI systems. It was built by a team of Meta researchers and presented in a paper dated 7 December 2023. The model is a Llama2-7b that has been instruction-tuned on a carefully collected but small dataset. Its job is to look at a user prompt and decide which safety risks it contains, and to do the same for the reply a language model produces. It works as a multi-class classifier and also emits binary decision scores, so it can be dropped into a moderation pipeline alongside a chat model.

How it came about

The work was led by Hakan Inan with ten co-authors, including Kartikeya Upasani, Jianfeng Chi and Madian Khabsa. It appeared on arXiv as submission 2312.06674 on 7 December 2023. The team says they released the model weights and invited researchers to build on and adapt them for AI safety work. No earlier or later versions of Llama Guard are described here.

Inside the model

The model starts from Llama2-7b and is instruction-tuned on a dataset the authors assembled for this task. Central to it is a safety risk taxonomy, a fixed set of categories covering risks found in prompts. The same taxonomy is applied to generated responses. Because the model behaves like an ordinary language model, the taxonomy can be swapped or extended at the input through zero-shot or few-shot prompting, and the output format can be changed to suit a given use case. The authors describe the tuning data as high quality but small in volume.

What it can do

On the OpenAI Moderation Evaluation dataset and on ToxicChat, the authors report results that match or exceed existing content moderation tools. They present the model as strong on those benchmarks despite the limited tuning data. Its outputs are multi-class labels plus binary safe or unsafe scores.

How to use it

The model weights are made available by the authors, who encourage researchers to develop and adapt them. The paper does not state the licence terms.

Safety and testing

Llama Guard is itself a safety tool: it is meant to screen what users send to an AI system and what the system sends back. The paper frames it as a safeguard for human-AI conversation and calls for the community to keep adapting it as AI safety needs change.

Questions people ask about Llama Guard

What does Llama Guard actually do?

It reads a prompt or a model response and labels which safety risks it contains, using a fixed taxonomy of categories. It also produces a binary safe or unsafe score, so it can act as a filter in front of or behind a chat model.

What is it built from?

It is a Llama2-7b model that has been instruction-tuned on a dataset collected specifically for this task. The authors describe that dataset as high quality but small in size.

Can I change the safety categories?

Yes. Because it behaves like a normal language model, you can supply a different taxonomy at the input through zero-shot or few-shot prompting, and adjust the output format to fit your use case.

How well does it perform?

The authors report that on the OpenAI Moderation Evaluation dataset and ToxicChat it matches or exceeds available content moderation tools. Those are the benchmarks given in the paper.

Is it available to use?

The model weights are released, and the researchers encourage others to build on and adapt them for AI safety. The paper does not specify the licence.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Llama Guard yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2312.06674. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.