News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
imgPhi-3

Phi-3

Small Microsoft language models built to run well on phones and laptops.

Phi-3 is a family of small language models from Microsoft, first shown in April 2024. The smallest, phi-3-mini, has 3.8 billion parameters and is trained on 3.3 trillion tokens, yet the team reports results close to much larger models. The line later grew to include 7B and 14B versions and a Phi-3.5 series with longer context, more languages and vision.

Share on WhatsApp

  • phi-3-mini: 3.8B parameters trained on 3.3 trillion tokens
  • Microsoft reports 69% on MMLU and 8.38 on MT-bench for phi-3-mini
  • Small enough to run on a phone, per the technical report
  • phi-3.5-MoE: 16 x 3.8B experts, 6.6B active parameters
  • Weights published openly, including ONNX and GGUF builds

What it is

Phi-3 is a group of compact language models released by Microsoft, aimed at devices with limited memory and computing power. The first and smallest member, phi-3-mini, carries 3.8 billion parameters and was trained on 3.3 trillion tokens. Microsoft says it can be placed on a phone. Larger siblings, phi-3-small at 7B and phi-3-medium at 14B, were trained on 4.8 trillion tokens and are described as clearly stronger. The models are meant for general use, with a particular emphasis on reasoning, maths and code, and are offered for both commercial and research work in English.

How it came about

The technical report was posted on 22 April 2024 and later revised, with the last version dated 30 August 2024. It carries a long author list led by Marah Abdin and colleagues at Microsoft, with Sebastien Bubeck as the submitting author. The work builds directly on phi-2, whose training set was scaled up for this release. A refreshed instruction-tuned mini model followed in June 2024, adding post-training data after customer feedback. The Phi-3.5 generation, covering mini, MoE and Vision variants, came later to widen language, multimodal and long-context coverage.

Inside the model

phi-3-mini holds 3.8 billion parameters and was trained on 3.3 trillion tokens drawn from heavily filtered public web pages plus synthetic data, a recipe scaled up from phi-2. The 7B and 14B models used 4.8 trillion tokens. After pre-training the mini model went through supervised fine-tuning and direct preference optimization, a method that tunes a model towards answers people prefer, to improve instruction following, robustness and safety. Its tokenizer has a vocabulary of up to 32,064 tokens, with placeholder tokens that can be extended. The Phi-3.5-MoE variant uses 16 experts of 3.8B each with 6.6 billion parameters active per token.

What it can do

Microsoft reports that phi-3-mini reached 69% on MMLU and 8.38 on MT-bench, results it says rival Mixtral 8x7B and GPT-3.5 while fitting on a phone. The 7B and 14B models scored 75% and 78% on MMLU and 8.7 and 8.9 on MT-bench. Phi-3.5-MoE is reported to beat comparable open models such as Llama 3.1 and the Mixtral line on language reasoning, maths and code, and to match Gemini-1.5-Flash and GPT-4o-mini. Phi-3.5-Vision handles single-image and multi-image prompts with text. The mini model is aimed mainly at English and at memory-limited settings.

How to use it

Weights are published openly for commercial and research use, with short and long context versions of the mini, small and medium models plus a vision model. Formats include Hugging Face, ONNX and GGUF for the mini. The models run in Azure AI Studio and work with version 4.41.2 of the transformers library. A cookbook and a portal page are provided. Microsoft asks developers to test the models in their own applications and to check accuracy, safety and fairness before deploying them.

Safety and testing

The mini model was aligned after pre-training for robustness, safety and chat format, using supervised fine-tuning and direct preference optimization. The June 2024 update added further post-training data and improved multi-turn conversation. The model card warns that the models were not designed or evaluated for every downstream purpose, and asks developers to assess and reduce risks around accuracy, safety and fairness, especially in high-risk uses, and to follow laws covering privacy and trade compliance.

Reception and debate

Microsoft describes adoption of the Phi-3 family as enthusiastic and says it continues to welcome community feedback. The June 2024 update was framed as a response to customer comments, with the company reporting sizeable gains on instruction following and structured output: JSON output rose from 11.5 to 52.3 and XML from 14.4 to 49.8 on its reported measures. No independent reviews, lawsuits or bans are described here.

Why it matters

Phi-3 pushed the idea that a model small enough for a phone could still handle serious reasoning tasks, and it fed into the Phi-3.5 line with mixture-of-experts and vision members. The team released several context lengths and export formats, and invited the community to work on conversions such as GGUF. The family continues the data-focused approach started with phi-2, where careful filtering and synthetic training data matter as much as raw model size.

Questions people ask about Phi-3

What is Phi-3?

Phi-3 is a set of small language models from Microsoft. The first, phi-3-mini, has 3.8 billion parameters and was trained on 3.3 trillion tokens. Larger 7B and 14B versions and a Phi-3.5 series followed, all aimed at devices with limited memory and compute.

How can it run on a phone?

The mini model is small at 3.8 billion parameters, and Microsoft designed the family for memory and compute constrained environments. The technical report states that phi-3-mini is small enough to be deployed on a phone while still reaching 69% on MMLU.

What data was it trained on?

The training set is a scaled-up version of the one used for phi-2, made of heavily filtered publicly available web data plus synthetic data. phi-3-mini saw 3.3 trillion tokens, while the 7B and 14B models were trained on 4.8 trillion tokens.

Can I use it commercially?

Yes. The model card states the models are intended for broad commercial and research use in English, and the weights are published openly. Microsoft asks developers to evaluate accuracy, safety and fairness for their own use case and to follow relevant laws.

What is different about Phi-3.5?

Phi-3.5 adds three models: mini, MoE and Vision. They target better multilingual, multimodal and long-context support. The MoE version uses 16 experts of 3.8B with 6.6B active parameters, and the 4.2B Vision model, derived from phi-3.5-mini, works with single or multiple images alongside text.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Phi-3 yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2404.14219, huggingface.co/microsoft/Phi-3-mini-4k-instruct. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.