Meditron
Open-source medical language models based on Llama-2.
Meditron is a suite of open-source large language models with 7 billion and 70 billion parameters, adapted for the medical domain. Released in November 2023 by academic researchers, it builds on Llama-2 and extends pretraining on curated medical data to improve access to medical knowledge.
- Released in November 2023 with 7B and 70B parameter versions.
- Built on Llama-2 with extended medical pretraining.
- Outperforms GPT-3.5 and Med-PaLM on medical benchmarks.
- Open-source weights and data curation code provided.
What it is
Meditron is a family of large language models designed specifically for the medical field. It includes versions with 7 billion and 70 billion parameters. The project aims to democratise access to medical knowledge by providing open-source alternatives to closed systems. These models are built upon the Llama-2 architecture but undergo additional pretraining on specialised medical data.
How it came about
The model was introduced in November 2023 through a research paper titled MEDITRON-70B: Scaling Medical Pretraining for Large Language Models. The work was conducted by a team of academics who sought to address the limitations of existing medical models, which were either proprietary or too small in scale. They released both the model weights and the code used to curate the training data.
Inside the model
The architecture relies on Llama-2, adapted using Nvidia’s Megatron-LM distributed trainer. The team extended the pretraining phase with a comprehensively curated medical corpus. This dataset includes selected articles and abstracts from PubMed as well as internationally recognised medical guidelines. This approach allows the model to learn domain-specific terminology and reasoning patterns beyond general language use.
What it can do
Evaluations across four major medical benchmarks showed significant performance gains over several baselines. The 70 billion parameter version achieved a 6 per cent absolute improvement over the best public baseline in its class. It also outperformed GPT-3.5 and Med-PaLM, while coming within 5 per cent of GPT-4 and 10 per cent of Med-PaLM-2 in comparative tests.
How to use it
The developers released the model weights and the code for curating the medical pretraining corpus. This open release is intended to drive further development of capable medical language models within the open-source community. Users can access the models without the restrictions typical of closed-source systems like GPT-4.
Why it matters
By providing a large-scale open-source option, Meditron challenges the dominance of closed medical models. It demonstrates that extensive domain-specific pretraining can yield competitive results against proprietary systems. The release of curation code supports reproducibility and encourages other researchers to build specialised models.
Questions people ask about Meditron
What is Meditron used for?
It is designed to provide access to medical knowledge and reasoning. The model helps users process medical information by leveraging a large corpus of PubMed articles and clinical guidelines.
Is Meditron open source?
Yes. The developers released both the model weights and the code used to curate the medical pretraining corpus. This allows the community to inspect, use, and build upon the work freely.
How does it compare to GPT-4?
The 70 billion parameter version performs within 5 per cent of GPT-4 on medical benchmarks. It outperforms GPT-3.5 and Med-PaLM, showing strong capabilities despite being smaller than some proprietary rivals.
Good reading and sources
There is no English Wikipedia article on Meditron yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2311.16079. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.