News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
Phi-1

Phi-1

A small Microsoft model specialised for basic Python coding

Phi-1 is a 1.3 billion parameter Transformer built by Microsoft for basic Python coding. It was trained for four days on eight A100 GPUs using selected web code and synthetic textbooks generated by GPT-3.5, and it scores above 50% on HumanEval despite its small size.

Share on WhatsApp

  • 1.3 billion parameter Transformer specialised for basic Python coding
  • Trained for 4 days on 8 A100 GPUs
  • 50.6% pass@1 on HumanEval, 55.5% on MBPP
  • Uses synthetic textbooks and exercises generated by GPT-3.5
  • Weights released under the MIT licence

What it is

Phi-1 is a language model for code, released by Microsoft in June 2023. It is a Transformer with 1.3 billion parameters, which is small next to most large language models of the time. The model is specialised for basic Python programming rather than general conversation. Its creators say it reaches over 50% accuracy on HumanEval, a benchmark of simple Python coding tasks. A smaller sibling, phi-1-small, has 350 million parameters and reaches 45% on the same benchmark. The work is presented as a research project, and the authors say the model is not intended for production coding.

How it came about

The model was introduced in the paper "Textbooks Are All You Need", submitted to arXiv on 20 June 2023 and revised on 2 October 2023. Nineteen authors are listed, led by Suriya Gunasekar at Microsoft. The paper presents phi-1 alongside phi-1-base, the model before fine-tuning on coding exercises, and phi-1-small, a 350 million parameter version trained with the same pipeline. The weights were published on Hugging Face under the MIT licence.

Inside the model

Phi-1 is a Transformer trained with a next-word prediction objective. Training ran for four days on eight A100 GPUs. The data combined about 6 billion tokens of "textbook quality" code and text selected from the web with 1 billion tokens of synthetic textbooks and exercises produced by GPT-3.5. Web sources included Python code from The Stack v1.2, question-and-answer content from StackOverflow, and competition code from code_contests. The model card reports 54 billion training tokens in total, of which 7 billion were unique. Fine-tuning used a dataset of coding exercises. Software used included PyTorch, DeepSpeed and Flash-Attention.

What it can do

The model scores 50.6% pass@1 on HumanEval and 55.5% on MBPP, benchmarks of simple Python tasks. The authors report surprising emergent behaviour after fine-tuning compared with phi-1-base. Limits are stated plainly: the model often produces incorrect code, its fine-tuning data uses only a handful of Python packages in 99.8% of scripts, and it may copy scripts found online. It answers chat or question-and-answer prompts unreliably and is weak at general conversation and common sense.

How to use it

Weights are downloadable from Hugging Face under the MIT licence. The model works with transformers version 4.37.0 or later. It is meant to be prompted in code format, with the model continuing after comments. The authors say generated code should be treated as a starting point and that production use is outside the scope of the project.

Safety and testing

The model card warns that generated code can contain security flaws, listing directory traversal, injection attacks, missing input validation, insecure defaults and poor error handling as examples. Users are told to review, test and verify code before deploying it, especially where security matters. The card also notes that web and synthetic training data can carry biases and errors that show up in outputs.

Why it matters

The paper's claim is that strong coding ability can come from a very small model trained on carefully chosen, textbook-like data, including synthetic data from another model. Phi-1 became an early example of the Phi family's approach of trading scale for data quality.

Questions people ask about Phi-1

What is Phi-1 for?

It is a code model focused on basic Python programming. Microsoft presents it as a research project, and the authors say it is not tested or intended for production-level coding. Its outputs should be treated as starting points rather than finished solutions.

How big is it and how was it trained?

Phi-1 has 1.3 billion parameters. Training took four days on eight A100 GPUs, using about 6 billion tokens of selected web data plus 1 billion tokens of synthetic textbooks and exercises made by GPT-3.5. The model card reports 54 billion training tokens in total, 7 billion of them unique.

How well does it perform?

The authors report 50.6% pass@1 on HumanEval and 55.5% on MBPP, both simple Python coding benchmarks. A smaller version, phi-1-small with 350 million parameters, reaches 45% on HumanEval.

Can I download it?

Yes. The weights are on Hugging Face under the MIT licence, and the model runs in the transformers library from version 4.37.0 onwards.

What are the main limits?

It frequently writes incorrect code, handles only a narrow set of Python packages, may reproduce scripts seen online, and answers chat-style prompts unreliably. The model card also warns that generated code can contain security weaknesses.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Phi-1 yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2306.11644, huggingface.co/microsoft/phi-1. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.