News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
Code Llama

Code Llama

Meta's code-specialised family of Llama 2 models in four sizes

Code Llama is a family of code-focused large language models released by Meta in August 2023, built by continuing to train Llama 2 on code data. Sizes run from 7B to 70B parameters, with Python and instruction-following variants, and the models are free for research and commercial use.

Share on WhatsApp

  • Four sizes: 7B, 13B, 34B and 70B parameters
  • Trained on 500B code tokens, 1T for the 70B model
  • Handles inputs up to 100,000 tokens
  • Code Llama 34B scored 53.7% on HumanEval, per Meta
  • Free for research and commercial use under the Llama 2 licence

What it is

Code Llama is a set of large language models made for programming work. Meta built it by taking Llama 2 and training it further on code and code-related text. It comes in four sizes: 7B, 13B, 34B and 70B parameters. Each size has three flavours: a base code model, a Python specialisation, and an Instruct version tuned to follow natural language instructions. It can write code from a description, complete code, debug it, and write natural language about code. Meta says it is aimed at professional developers and people learning to code.

How it came about

Meta published Code Llama on 24 August 2023, along with a research paper listing 26 authors led by Baptiste Rozière. The paper was revised twice, most recently on 31 January 2024. On 29 January 2024 Meta added the 70B version, described as the largest and best-performing model in the family. It arrived in the same three variants as the earlier sizes, all under the Llama 2 community licence.

Inside the model

Code Llama starts from Llama 2 and is trained further by sampling more from code-specific datasets. The 7B, 13B and 34B models see 500B tokens of code and code-related data; the 70B model sees 1T. Training uses sequences of 16,000 tokens, with gains reported on inputs up to 100,000 tokens. The Python variant is fine-tuned on a further 100B tokens of Python. The Instruct variant is instruction-tuned and aligned to give helpful, safe answers. The 7B and 13B base and instruct models also learn fill-in-the-middle, so they can insert code into existing code.

What it can do

The models handle popular languages including Python, C++, Java, PHP, TypeScript (JavaScript), C# and Bash. Meta reports Code Llama 34B scoring 53.7% on HumanEval and 56.2% on MBPP, which it describes as on par with ChatGPT. The paper reports scores up to 67% on HumanEval and 65% on MBPP, and says Code Llama - Python 7B beats Llama 2 70B on both. Meta advises against using the base and Python models for general natural language tasks, since they are not built to follow instructions.

How to use it

Weights are downloadable under the Llama 2 community licence, free for research and commercial use, with an acceptable use policy attached. Training recipes are on Meta's GitHub repository. The 7B model can run on a single GPU; the 34B and 70B give the best results, while 7B and 13B suit low-latency work such as real-time completion.

Safety and testing

Before release Meta ran red teaming with specialists in responsible AI, offensive security, malware development and software engineering. A quantitative test used prompts clearly asking for malicious code and scored the replies against ChatGPT (GPT-3.5 Turbo); Meta says its model answered more safely. The company's Responsible Use Guide covers content policies, evaluation, input and output risks, and transparency, and it recommends code-specific safety studies and adversarial red teaming.

Why it matters

Meta positions Code Llama as a base for others to build on, hoping it spurs new research and commercial tools built on Llama 2. The company argues that open code models let the wider community check capabilities, spot problems and fix weaknesses. It says the models are meant for software engineers across research, industry, open source projects, NGOs and businesses.

Questions people ask about Code Llama

What is Code Llama?

It is a family of code-focused large language models from Meta, built by training Llama 2 further on code data. It writes, completes and debugs code, and can explain code in natural language. Sizes range from 7B to 70B parameters.

What are the different versions?

Each size comes in three flavours. The base Code Llama handles code tasks. Code Llama - Python is fine-tuned on 100B more Python tokens. Code Llama - Instruct is tuned to follow natural language instructions, and Meta recommends it for code generation.

Can I use it commercially?

Yes. Meta released it under the same community licence as Llama 2, free for research and commercial use. Users must follow the licence and the acceptable use policy.

How much code can it take at once?

Training used sequences of 16,000 tokens, but the models show improvements on inputs up to 100,000 tokens. That lets developers pass in large chunks of a codebase for context or debugging.

Is it safe to ask it for malicious code?

Meta ran red teaming and a quantitative test with prompts clearly requesting malicious code, scoring replies against ChatGPT (GPT-3.5 Turbo). It reports safer responses from its own model, and advises developers to do their own code-specific safety studies.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Code Llama yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2308.12950, ai.meta.com/blog/code-llama-large-language-model-coding/. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.