BioGPT
A generative Transformer trained on biomedical writing for text and mining tasks
BioGPT is a language model that writes and analyses biomedical text, built by Microsoft researchers and described in a paper first posted in October 2022. It was trained on a large body of biomedical literature and reported strong results on six language tasks in that field, including relation extraction and question answering.
- Built by Microsoft researchers and first posted as a preprint in October 2022
- Trained on a large body of biomedical literature to write and mine text
- F1 of 44.98% on BC5CDR relation extraction, as reported by the team
- 78.2% accuracy on PubMedQA, described as a new record at the time
- Weights published on Hugging Face as microsoft/biogpt
What it is
BioGPT is a language model aimed squarely at life-science writing. A team from Microsoft built it and shared the work through a preprint in October 2022, later published in Briefings in Bioinformatics. Unlike earlier biomedical systems in the BERT family, which sort or label text, this one can also produce new sentences. The authors trained it on a large collection of biomedical literature so it could both mine facts from papers and write fluent descriptions of medical terms. They tested it on six language-processing tasks in the biomedical field and reported that it beat earlier systems on most of them.
How it came about
The work was led by Renqian Luo with Liai Sun, Yingce Xia, Tao Qin, Sheng Zhang, Hoifung Poon and Tie-Yan Liu. It appeared on arXiv on 19 October 2022, with a revised version in April 2023. The paper was published in Briefings in Bioinformatics under the identifier bbac409. The team framed it as an answer to a gap: biomedical models of the BERT type handled classification well but could not generate text, which limited what they could be used for.
Inside the model
BioGPT follows the GPT pattern, a Transformer that predicts the next piece of text, rather than the BERT pattern used by earlier biomedical systems. Its training corpus was large-scale biomedical literature, so its vocabulary and knowledge lean towards medicine and biology. The published description does not give layer counts, parameter totals, context window or training compute. The team reports that the model can produce fluent wording for biomedical terms, and they released code alongside the paper. Weights are hosted on Hugging Face under microsoft/biogpt and can be loaded through the Transformers library with a BioGPT tokenizer and causal language-model class.
What it can do
The team evaluated the model on six biomedical language tasks. On end-to-end relation extraction they report F1 scores of 44.98% for BC5CDR, 38.42% for KD-DTI and 40.76% for DDI. On PubMedQA, a question-answering set built from PubMed abstracts, they report 78.2% accuracy and describe it as a new record at the time. A case study of generation showed the model writing readable descriptions of biomedical terms. The published pages do not set out limits or failure cases.
How to use it
The weights are published on Hugging Face as microsoft/biogpt and can be downloaded and run locally. The page shows how to load them with the Transformers library for text generation and for extracting features from a passage. Code accompanies the paper. The pages reviewed do not state the licence terms for the weights.
Why it matters
BioGPT is presented as a step beyond earlier biomedical models that could only classify text, showing that a generative system trained on life-science papers could both mine relations and write descriptions. Its reported scores on relation extraction and PubMedQA were put forward as new records at the time of publication. The paper and code are public, and the weights are downloadable, so others can build on the system.
Questions people ask about BioGPT
What is BioGPT for?
It handles biomedical language tasks: pulling relations such as drug interactions out of papers, answering questions from PubMed abstracts, and writing fluent descriptions of medical terms. The team tested it on six tasks in this area.
Who made it?
Renqian Luo and colleagues at Microsoft. Their paper appeared on arXiv in October 2022 and was later published in Briefings in Bioinformatics.
How is it different from BioBERT and PubMedBERT?
Those earlier systems follow the BERT pattern and are good at labelling and sorting text but cannot write new text. BioGPT follows the GPT pattern, so it generates language as well as mining it.
How well does it score?
The team reports F1 scores of 44.98% on BC5CDR, 38.42% on KD-DTI and 40.76% on DDI for relation extraction, plus 78.2% accuracy on PubMedQA, which they describe as a new record at the time.
Can I run it myself?
Yes. The weights are on Hugging Face as microsoft/biogpt and load through the Transformers library with a BioGPT tokenizer and causal language-model class, for text generation or feature extraction.
Good reading and sources
There is no English Wikipedia article on BioGPT yet, so the facts on this page come from the sources listed above: huggingface.co/microsoft/biogpt, arxiv.org/abs/2210.10341. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.