News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
Phi-4

Phi-4

A 14B Microsoft Research model trained heavily on synthetic data for reasoning.

Phi-4 is a 14-billion parameter language model from Microsoft Research, released on 12 December 2024. It leans on synthetic, textbook-style training data rather than mostly web content, and the team says it beats its GPT-4 teacher on STEM question answering despite small changes to the Phi-3 architecture.

Share on WhatsApp

  • 14 billion parameters, dense decoder-only Transformer
  • Trained on 9.8T tokens over 21 days, Oct-Nov 2024
  • Released 12 December 2024 by Microsoft Research
  • Synthetic textbook-like data is central to the training recipe
  • Scores 56.1 on GPQA and 80.4 on MATH in the model card's table

What it is

Phi-4 is a 14-billion parameter language model built by Microsoft Research and put out on 12 December 2024. It is a dense decoder-only Transformer, meaning every part of the network is used for each token it produces. The team designed it for research and as a building block in generative AI features, mainly for English tasks where memory, compute or response speed are tight, and where reasoning and logic matter. Training ran from October to November 2024 over 9.8 trillion tokens in 21 days. It is a static model, with training data cut off at June 2024 or earlier.

How it came about

Phi-4 continues the Phi line from Microsoft Research, whose earlier models largely distilled abilities from a teacher model, specifically GPT-4. The technical report, by Marah Abdin and 26 co-authors, appeared on arXiv on 12 December 2024. The team kept the Phi-3 architecture almost unchanged and instead pushed on data, the training curriculum and post-training. They report that the model goes well beyond its teacher on STEM question answering, which they take as evidence that their data generation and post-training methods are doing more than copying GPT-4.

Inside the model

The model is a dense decoder-only Transformer with 14 billion parameters. Its training mix extends what Phi-3 used: rigorously filtered public documents, selected educational data and code, plus acquired academic books and Q&A sets. A large share is newly written synthetic, textbook-like material meant to teach maths, coding, common-sense reasoning and world knowledge, including science, daily activities and theory of mind. Multilingual data is about 8% of the total. After pre-training, the model went through supervised fine-tuning and iterative direct preference optimisation, a method that tunes a model towards answers people prefer, for instruction following and safety.

What it can do

On OpenAI's SimpleEval suite, phi-4 (14B) scored 84.8 on MMLU, 56.1 on GPQA graduate-level science, 80.4 on MATH, 82.6 on HumanEval code generation, 75.5 on DROP and 3.0 on SimpleQA factual questions. The model card places it above Phi-3 (14B) on most of these and above GPT-4o on GPQA and MATH, while GPT-4o leads on MMLU, HumanEval and SimpleQA. The card notes Llama-3.3-70B scores may be understated by the strict formatting rules of simple-evals; Meta reports 77 for MATH and 88 for HumanEval on that model.

How to use it

The weights are published on Hugging Face under the name microsoft/phi-4. The model card shows usage through the transformers pipeline with chat-format prompts, which suit it best given its training data. The card does not state the licence terms.

Safety and testing

Safety work combined supervised fine-tuning with iterative direct preference optimisation, using open-source and in-house synthetic sets covering helpfulness, harmlessness and multiple safety categories. Before release, the team ran quantitative tests with open-source safety benchmarks and adversarial conversation simulation. Microsoft's independent AI Red Team assessed risks in ordinary and adversarial settings, covering jailbreaks, encoding-based attacks, multi-turn attacks and adversarial suffix attacks.

Reception and debate

The model card itself lists limits: it is trained mainly on English and is not intended for multilingual use, other languages and less represented English varieties will do worse, and it can produce stereotypes, offensive content or fabricated facts. Code support is narrow, with most training code in Python and common packages. No external reception is described here.

Why it matters

The team argues that beating the GPT-4 teacher on STEM question answering shows their data-generation and post-training methods go past simple distillation. They also say the gains come with only minimal changes to the Phi-3 architecture, pointing to data quality and training design as the main levers for small, capable models.

Questions people ask about Phi-4

Who made Phi-4 and when did it come out?

Microsoft Research developed it. The technical report and the model release both appeared on 12 December 2024, with training carried out between October and November 2024.

What makes its training data different?

Rather than leaning mostly on web content or code, the team used a large amount of newly created synthetic, textbook-like data aimed at teaching maths, coding, common sense and world knowledge, alongside filtered public documents and acquired academic books and Q&A sets.

How big is it and what can it handle?

It has 14 billion parameters and a context window of 16K tokens. It takes text in and produces text, and works best with prompts written in a chat format.

Is it good at languages other than English?

No. About 8% of the training data is multilingual, but the model card says it is not intended to support multilingual use and that other languages will get worse performance than English.

How do I get it?

The weights are published on Hugging Face as microsoft/phi-4, and the model card shows how to run it with the transformers pipeline. The card does not state the licence terms.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Phi-4 yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2412.08905, huggingface.co/microsoft/phi-4. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.