Phi-1
A small Microsoft model specialised for basic Python coding
Phi-1 is a 1.3 billion parameter Transformer built by Microsoft for basic Python coding. It was trained for four days on eight A100 GPUs using selected web code and synthetic textbooks generated by GPT-3.5, and it scores above 50% on HumanEval despite its small size.
- 1.3 billion parameter Transformer specialised for basic Python coding
- Trained for 4 days on 8 A100 GPUs
- 50.6% pass@1 on HumanEval, 55.5% on MBPP
- Uses synthetic textbooks and exercises generated by GPT-3.5
- Weights released under the MIT licence
What it is
Phi-1 is a language model for code, released by Microsoft in June 2023. It is a Transformer with 1.3 billion parameters, which is small next to most large language models of the time. The model is specialised for basic Python programming rather than general conversation. Its creators say it reaches over 50% accuracy on HumanEval, a benchmark of simple Python coding tasks. A smaller sibling, phi-1-small, has 350 million parameters and reaches 45% on the same benchmark. The work is presented as a research project, and the authors say the model is not intended for production coding.
How it came about
The model was introduced in the paper "Textbooks Are All You Need", submitted to arXiv on 20 June 2023 and revised on 2 October 2023. Nineteen authors are listed, led by Suriya Gunasekar at Microsoft. The paper presents phi-1 alongside phi-1-base, the model before fine-tuning on coding exercises, and phi-1-small, a 350 million parameter version trained with the same pipeline. The weights were published on Hugging Face under the MIT licence.
Inside the model
Phi-1 is a Transformer trained with a next-word prediction objective. Training ran for four days on eight A100 GPUs. The data combined about 6 billion tokens of "textbook quality" code and text selected from the web with 1 billion tokens of synthetic textbooks and exercises produced by GPT-3.5. Web sources included Python code from The Stack v1.2, question-and-answer content from StackOverflow, and competition code from code_contests. The model card reports 54 billion training tokens in total, of which 7 billion were unique. Fine-tuning used a dataset of coding exercises. Software used included PyTorch, DeepSpeed and Flash-Attention.
What it can do
The model scores 50.6% pass@1 on HumanEval and 55.5% on MBPP, benchmarks of simple Python tasks. The authors report surprising emergent behaviour after fine-tuning compared with phi-1-base. Limits are stated plainly: the model often produces incorrect code, its fine-tuning data uses only a handful of Python packages in 99.8% of scripts, and it may copy scripts found online. It answers chat or question-and-answer prompts unreliably and is weak at general conversation and common sense.
How to use it
Weights are downloadable from Hugging Face under the MIT licence. The model works with transformers version 4.37.0 or later. It is meant to be prompted in code format, with the model continuing after comments. The authors say generated code should be treated as a starting point and that production use is outside the scope of the project.
Safety and testing
The model card warns that generated code can contain security flaws, listing directory traversal, injection attacks, missing input validation, insecure defaults and poor error handling as examples. Users are told to review, test and verify code before deploying it, especially where security matters. The card also notes that web and synthetic training data can carry biases and errors that show up in outputs.
Why it matters
The paper's claim is that strong coding ability can come from a very small model trained on carefully chosen, textbook-like data, including synthetic data from another model. Phi-1 became an early example of the Phi family's approach of trading scale for data quality.
Questions people ask about Phi-1
What is Phi-1 for?
It is a code model focused on basic Python programming. Microsoft presents it as a research project, and the authors say it is not tested or intended for production-level coding. Its outputs should be treated as starting points rather than finished solutions.
How big is it and how was it trained?
Phi-1 has 1.3 billion parameters. Training took four days on eight A100 GPUs, using about 6 billion tokens of selected web data plus 1 billion tokens of synthetic textbooks and exercises made by GPT-3.5. The model card reports 54 billion training tokens in total, 7 billion of them unique.
How well does it perform?
The authors report 50.6% pass@1 on HumanEval and 55.5% on MBPP, both simple Python coding benchmarks. A smaller version, phi-1-small with 350 million parameters, reaches 45% on HumanEval.
Can I download it?
Yes. The weights are on Hugging Face under the MIT licence, and the model runs in the transformers library from version 4.37.0 onwards.
What are the main limits?
It frequently writes incorrect code, handles only a narrow set of Python packages, may reproduce scripts seen online, and answers chat-style prompts unreliably. The model card also warns that generated code can contain security weaknesses.
Good reading and sources
There is no English Wikipedia article on Phi-1 yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2306.11644, huggingface.co/microsoft/phi-1. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.