Med-PaLM 2
Google's medical question-answering model built on PaLM 2
Med-PaLM 2 is a medical large language model from Google, described in a May 2023 paper by Karan Singhal and colleagues. It combines the PaLM 2 base model with medical fine-tuning and prompting strategies, and reached up to 86.5% on the MedQA benchmark, well above the earlier Med-PaLM.
- Scored up to 86.5% on MedQA, over 19% above Med-PaLM
- Built on PaLM 2 with medical fine-tuning and ensemble refinement prompting
- Physicians preferred its answers on 8 of 9 clinical utility axes
- Tested on 240 adversarial long-form medical questions
- Presented in arXiv preprint 2305.09617, 16 May 2023
What it is
Med-PaLM 2 is a large language model aimed at answering medical questions at a level comparable to physicians. Google researchers presented it in a paper posted on 16 May 2023. It builds on PaLM 2, the company's general-purpose model, adding medical domain fine-tuning and prompting techniques. The team reports a score of up to 86.5% on MedQA, a dataset of US Medical Licensing Examination style questions, which is more than 19 percentage points above the earlier Med-PaLM. The work targets consumer medical questions and long-form clinical answers, with physicians judging the output against answers written by clinicians.
How it came about
The model follows Med-PaLM, which the same group describes as the first system to pass the threshold score on USMLE-style questions, at 67.2% on MedQA. The Med-PaLM 2 paper lists 31 authors, led by Karan Singhal and Tao Tu, with Shekoofeh Azizi and Vivek Natarajan among the senior contributors. It appeared as arXiv preprint 2305.09617 on 16 May 2023. The paper notes that further studies are needed to check how such models perform in real clinical settings.
Inside the model
Three ingredients make up the system, according to the authors: improvements in the underlying PaLM 2 model, fine-tuning on medical data, and prompting strategies. One prompting method is new and called ensemble refinement, where the model is prompted to refine its own answers. The paper does not give parameter counts, layer counts, training data volume or compute figures. Human evaluation covered long-form answers along several axes relevant to clinical use, including a set of 240 adversarial questions built to expose weaknesses in language models.
What it can do
On MedQA the model reached up to 86.5%, a gain of over 19% on Med-PaLM. The team also reports results near or above the best published figures on MedMCQA, PubMedQA and the clinical topics of MMLU. In pairwise comparisons over 1066 consumer medical questions, physicians preferred Med-PaLM 2 answers to physician-written answers on eight of nine axes of clinical utility, with p < 0.001. On the 240 adversarial questions the model improved on Med-PaLM across every axis measured, also at p < 0.001. The authors stress that real-world clinical effectiveness remains untested.
How to use it
The paper describes research results only. It does not state that weights are released, nor does it name a licence, an app or an API for the model.
Safety and testing
Testing included a purpose-built set of 240 long-form adversarial questions designed to probe where language models fail, and the team measured gains on these against Med-PaLM. Physicians took part in the evaluations. The authors say more work is required before the model can be relied on in practice.
Reception and debate
The paper reports physician preference for the model's answers over clinician-written ones on most clinical utility axes. It also states plainly that validation in real-world settings has not yet been done. No lawsuits, bans or public incidents are described.
Why it matters
Med-PaLM 2 pushed medical question answering well past the earlier Med-PaLM result on MedQA and, by the authors' account, set a new best figure on that dataset at the time. The evaluation method, pairing benchmark scores with physician review of long-form answers and adversarial probing, is presented as a step towards physician-level performance in this task.
Questions people ask about Med-PaLM 2
What is Med-PaLM 2?
It is a medical large language model from Google, built by fine-tuning PaLM 2 for medical question answering and adding prompting strategies. The researchers describe it in a paper from May 2023, where it scores up to 86.5% on MedQA, a set of US medical licensing style questions.
How is it different from Med-PaLM?
It uses a stronger base model, PaLM 2, plus medical domain fine-tuning and a prompting method called ensemble refinement. On MedQA it scores up to 86.5%, more than 19 percentage points above Med-PaLM's 67.2%, and physicians rated it higher on every axis of the adversarial question set.
Can I download or use it?
The paper reports research results and does not announce released weights, a licence, an app or an API. Nothing in the description offers the model for public use.
Is it as good as a doctor?
In pairwise reviews of 1066 consumer medical questions, physicians preferred its answers to physician-written ones on eight of nine clinical utility axes. The authors caution that real-world clinical effectiveness still needs to be validated in further studies.
Good reading and sources
There is no English Wikipedia article on Med-PaLM 2 yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2305.09617. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.