Code models
Codex, Code Llama, StarCoder, Qwen Coder and Codestral. 53 models on EveryLLM so far.
Code models are language models trained or tuned to read and write software. They complete a line as a programmer types, turn a plain-English request into a function, explain unfamiliar code, write tests and, in their newest form, act as agents that edit a whole repository and run its tests. Source code is unusually good training data: it is plentiful on the public web, highly structured and easy to check by running it.
From Codex to Copilot
OpenAI's Codex, described in a July 2021 paper, was a GPT-3 model fine-tuned on public code from GitHub. Fine-tuning means further training a finished model on a narrower dataset. Codex powered GitHub Copilot, which opened as a preview in June 2021. The same paper introduced HumanEval, 164 hand-written programming problems checked by unit tests, which became the first standard yardstick for code models.
Open code models
Open-weights code models followed quickly. BigCode, a project led by Hugging Face and ServiceNow, released StarCoder in May 2023, trained on permissively licensed code in more than 80 programming languages. Meta released Code Llama in August 2023, and Mistral's Codestral, DeepSeek-Coder, IBM's Granite code models and Alibaba's Qwen Coder series followed. Many now handle hundreds of thousands of tokens, enough to read a sizeable codebase at once.
Agents that write software
Since 2024 the focus has moved from completing lines to finishing tasks. SWE-bench, released by Princeton researchers in October 2023, asks a model to fix real issues from GitHub projects. Coding agents such as Anthropic's Claude Code and OpenAI's Codex agent, both launched in 2025, read files, run commands and propose changes. Open questions remain over the licensing of training code, the security of generated code and how much human review is still needed.
Famous code models
Still to be written: GitHub Copilot, OpenAI Codex.