News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
Med-PaLM 2

Med-PaLM 2

Google's medical question-answering model built on PaLM 2

Med-PaLM 2 is a medical large language model from Google, described in a May 2023 paper by Karan Singhal and colleagues. It combines the PaLM 2 base model with medical fine-tuning and prompting strategies, and reached up to 86.5% on the MedQA benchmark, well above the earlier Med-PaLM.

Share on WhatsApp

  • Scored up to 86.5% on MedQA, over 19% above Med-PaLM
  • Built on PaLM 2 with medical fine-tuning and ensemble refinement prompting
  • Physicians preferred its answers on 8 of 9 clinical utility axes
  • Tested on 240 adversarial long-form medical questions
  • Presented in arXiv preprint 2305.09617, 16 May 2023

What it is

Med-PaLM 2 is a large language model aimed at answering medical questions at a level comparable to physicians. Google researchers presented it in a paper posted on 16 May 2023. It builds on PaLM 2, the company's general-purpose model, adding medical domain fine-tuning and prompting techniques. The team reports a score of up to 86.5% on MedQA, a dataset of US Medical Licensing Examination style questions, which is more than 19 percentage points above the earlier Med-PaLM. The work targets consumer medical questions and long-form clinical answers, with physicians judging the output against answers written by clinicians.

How it came about

The model follows Med-PaLM, which the same group describes as the first system to pass the threshold score on USMLE-style questions, at 67.2% on MedQA. The Med-PaLM 2 paper lists 31 authors, led by Karan Singhal and Tao Tu, with Shekoofeh Azizi and Vivek Natarajan among the senior contributors. It appeared as arXiv preprint 2305.09617 on 16 May 2023. The paper notes that further studies are needed to check how such models perform in real clinical settings.

Inside the model

Three ingredients make up the system, according to the authors: improvements in the underlying PaLM 2 model, fine-tuning on medical data, and prompting strategies. One prompting method is new and called ensemble refinement, where the model is prompted to refine its own answers. The paper does not give parameter counts, layer counts, training data volume or compute figures. Human evaluation covered long-form answers along several axes relevant to clinical use, including a set of 240 adversarial questions built to expose weaknesses in language models.

What it can do

On MedQA the model reached up to 86.5%, a gain of over 19% on Med-PaLM. The team also reports results near or above the best published figures on MedMCQA, PubMedQA and the clinical topics of MMLU. In pairwise comparisons over 1066 consumer medical questions, physicians preferred Med-PaLM 2 answers to physician-written answers on eight of nine axes of clinical utility, with p < 0.001. On the 240 adversarial questions the model improved on Med-PaLM across every axis measured, also at p < 0.001. The authors stress that real-world clinical effectiveness remains untested.

How to use it

The paper describes research results only. It does not state that weights are released, nor does it name a licence, an app or an API for the model.

Safety and testing

Testing included a purpose-built set of 240 long-form adversarial questions designed to probe where language models fail, and the team measured gains on these against Med-PaLM. Physicians took part in the evaluations. The authors say more work is required before the model can be relied on in practice.

Reception and debate

The paper reports physician preference for the model's answers over clinician-written ones on most clinical utility axes. It also states plainly that validation in real-world settings has not yet been done. No lawsuits, bans or public incidents are described.

Why it matters

Med-PaLM 2 pushed medical question answering well past the earlier Med-PaLM result on MedQA and, by the authors' account, set a new best figure on that dataset at the time. The evaluation method, pairing benchmark scores with physician review of long-form answers and adversarial probing, is presented as a step towards physician-level performance in this task.

Questions people ask about Med-PaLM 2

What is Med-PaLM 2?

It is a medical large language model from Google, built by fine-tuning PaLM 2 for medical question answering and adding prompting strategies. The researchers describe it in a paper from May 2023, where it scores up to 86.5% on MedQA, a set of US medical licensing style questions.

How is it different from Med-PaLM?

It uses a stronger base model, PaLM 2, plus medical domain fine-tuning and a prompting method called ensemble refinement. On MedQA it scores up to 86.5%, more than 19 percentage points above Med-PaLM's 67.2%, and physicians rated it higher on every axis of the adversarial question set.

Can I download or use it?

The paper reports research results and does not announce released weights, a licence, an app or an API. Nothing in the description offers the model for public use.

Is it as good as a doctor?

In pairwise reviews of 1066 consumer medical questions, physicians preferred its answers to physician-written ones on eight of nine clinical utility axes. The authors caution that real-world clinical effectiveness still needs to be validated in further studies.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Med-PaLM 2 yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2305.09617. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.