News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
Phi-4-mini

Phi-4-mini

A 3.8B-parameter compact language model from Microsoft for reasoning, maths and code

Phi-4-mini is a compact language model with 3.8 billion parameters, built by Microsoft and described in a technical report submitted in March 2025. Trained on curated web and synthetic data with a maths and code focus, it matches larger models on reasoning tasks and is offered as open weights for multilingual use.

Share on WhatsApp

  • 3.8 billion parameters, released by Microsoft in March 2025
  • Vocabulary of 200,064 tokens for multilingual support
  • 128K-token context length in the instruct release
  • Scores 88.6 on GSM8K and 64.0 on MATH in Microsoft's tests
  • Trained on curated web and synthetic data focused on maths and code

What it is

Phi-4-mini is a small language model with 3.8 billion parameters, released by Microsoft as part of the Phi-4 family. It is a text-only model aimed at settings where memory and compute are limited, and at tasks needing strong reasoning, especially maths and logic. Microsoft says it was trained on curated web data plus synthetic data, with an emphasis on high-quality maths and coding material. The instruct version supports a 128K-token context window and a vocabulary of about 200,000 tokens, which helps with multilingual use. The company positions it for broad commercial and research use, and as a building block for generative AI features.

How it came about

The model was presented in the Phi-4-Mini technical report, first posted on arXiv on 3 March 2025 and revised on 7 March 2025, with 74 Microsoft authors. Microsoft says the release builds on feedback from the Phi-3 series. The report also covers Phi-4-Multimodal, a sibling model adding vision and speech, and an experimental reasoning-focused version of Phi-4-mini. Related checkpoints include Phi-4-mini-instruct, Phi-4-mini-reasoning and ONNX exports.

Inside the model

Phi-4-mini is a decoder-only transformer with group query attention, which makes long-sequence generation more efficient. Its vocabulary was expanded to 200,064 tokens, up from the earlier Phi-3.5-Mini, to better handle multiple languages. Training used a carefully curated mix of web data and synthetic data, weighted towards maths and coding problems that demand multi-step reasoning. Post-training combined supervised fine-tuning with direct preference optimization, a method that trains the model on pairs of preferred and rejected answers, to improve instruction following, function calling and safety behaviour. The model card notes placeholder tokens in the tokenizer that can be extended for downstream fine-tuning.

What it can do

Microsoft reports that Phi-4-mini outperforms open models of similar size and matches models twice its size on maths and coding. In its benchmark table, it scores 88.6 on GSM8K and 64.0 on MATH with chain-of-thought prompting, ahead of Phi-3.5-mini-instruct at 76.9 and 49.8. Its overall aggregate is 63.5, versus 60.5 for Phi-3.5-mini and 67.9 for Qwen2.5-7B-Instruct. Multilingual MGSM reaches 63.9. The company also says an experimental reasoning version performs on par with or better than DeepSeek-R1-Distill-Qwen-7B and DeepSeek-R1-Distill-Llama-8B. Limits are stated plainly: at 3.8B parameters the model cannot store much factual knowledge and may give factually wrong answers, though retrieval-augmented use can help.

How to use it

The weights are published openly on Hugging Face as microsoft/Phi-4-mini-instruct, with an ONNX variant for optimised runtimes. Microsoft offers a demo space on Hugging Face and access through Azure. The model card describes the release as open and permissive, and lists a Phi Cookbook and Phi Portal as supporting resources. It runs with vLLM and Hugging Face Transformers.

Safety and testing

Microsoft says the model went through supervised fine-tuning and direct preference optimization partly to support robust safety measures. The model card warns that the model was not evaluated for every downstream purpose, that performance varies across languages, and that developers should assess accuracy, safety and fairness before deploying it, particularly in high-risk settings. It also reminds users to follow applicable laws, including privacy and trade compliance rules.

Reception and debate

The technical report claims strong results against similarly sized and larger open models on maths, coding and reasoning benchmarks, and a first-place position for the sibling Phi-4-Multimodal on the OpenASR leaderboard at the time of writing. No independent reviews, disputes or bans are described in the available material.

Why it matters

Phi-4-mini extends Microsoft's Phi line of small models, showing that careful data curation can let a 3.8B-parameter model compete with models twice its size on reasoning-heavy tasks. Its larger vocabulary and group query attention were adopted to serve multilingual and long-context use, and the accompanying Phi-4-Multimodal work introduced a mixture-of-LoRAs approach for adding vision and speech to a compact base model.

Questions people ask about Phi-4-mini

How big is Phi-4-mini?

It has 3.8 billion parameters. Microsoft describes it as compact and suited to memory- and compute-constrained environments, and says it can match models twice its size on maths and coding tasks that need complex reasoning.

Who made it and when did it come out?

Microsoft built it. The technical report was first posted to arXiv on 3 March 2025, with a revised version on 7 March 2025, listing 74 Microsoft authors. The instruct checkpoint is available on Hugging Face and Azure.

What can it do?

It handles general multilingual text tasks, with particular strength in maths, coding and reasoning. Microsoft's tests report 88.6 on GSM8K and 64.0 on MATH, and it supports function calling through a tool-enabled prompt format.

Is it open?

Yes. The weights are published on Hugging Face under a permissive MIT licence, with an ONNX version also available. Microsoft says it is intended for broad commercial and research use.

What are its limits?

Microsoft states that at 3.8B parameters the model cannot hold much factual knowledge, so it may produce incorrect facts. The company suggests pairing it with a search engine or retrieval-augmented generation to reduce that weakness.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Phi-4-mini yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2503.01743, huggingface.co/microsoft/Phi-4-mini-instruct. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.