News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
Galactica

Galactica

A language model trained on scientific text for research tasks

Galactica is a family of language models built for scientific work, trained on papers, textbooks, encyclopaedias and knowledge bases. The Papers with Code team at Meta AI released it in November 2022, with sizes from 125 million to 120 billion parameters, and published the weights for non-commercial use.

Share on WhatsApp

  • Trained on 106 billion tokens of open-access scientific text and data
  • Model sizes run from 125 million to 120 billion parameters
  • Scored 68.2% on LaTeX equation probes versus GPT-3's 49.0%
  • Weights released under a non-commercial CC BY-NC 4.0 licence
  • Built by the Papers with Code team at Meta AI, November 2022

What it is

Galactica is a language model aimed squarely at scientific work. The Papers with Code group inside Meta AI built it to see whether such a system could help organise scientific knowledge rather than just retrieve it. Released in November 2022, the family spans sizes from 125 million up to 120 billion parameters. It was trained on open-access scientific writing and data, and is meant for tasks such as predicting citations, answering scientific questions, doing mathematical reasoning, summarising documents, writing papers, estimating molecular properties and pulling out named entities. The team says the goal was to give science a new kind of interface.

How it came about

The work came out of Meta AI, where the Papers with Code team studied how language models could organise scientific knowledge automatically. A paper describing the model appeared on arXiv on 16 November 2022, written by Ross Taylor and colleagues including Robert Stojnic. The authors released the model openly for the scientific community and put up a public demo alongside the paper. Model checkpoints were later added to the Hugging Face Hub by Manuel Romero.

Inside the model

Galactica uses a decoder-only transformer, meaning it predicts text forwards, with some changes described in the release paper. Training data covered 106 billion tokens drawn from open-access papers, textbooks, scientific websites, encyclopaedias, reference works and knowledge bases. Different kinds of scientific content were turned into tokens so the model could handle several task types through plain language. The released checkpoints range from 125 million to 120 billion parameters. The largest version runs through the transformers library, with options for reduced precision on GPU.

What it can do

The team reports gains over earlier models on knowledge probes, reasoning and knowledge-heavy scientific tasks. On LaTeX equation probes it scored 68.2% against GPT-3's 49.0%. In mathematical MMLU it reached 41.3% versus Chinchilla's 35.7%, and on MATH it scored 20.4% against PaLM 540B's 8.8%. It set results of 77.6% on PubMedQA and 52.9% on MedMCQA dev. Despite no general-purpose training text, it also beat BLOOM and OPT-175B on BIG-bench. Limits remain: the model invents content, especially around obscure concepts, and its citation choices still favour popular work.

How to use it

Weights are downloadable under a Creative Commons BY-NC 4.0 licence, which bars commercial use. The model card warns against production deployment without safeguards because of hallucination risk. Example scripts show how to load the 120 billion parameter checkpoint on CPU or GPU, including half precision and 8-bit settings. A demo and the release paper were published alongside the model.

Safety and testing

The team evaluated the model on benchmarks covering stereotypes and toxicity. They report substantially lower toxicity rates than other large language models, while noting that bias still shows up on some measures. The model card cautions that outputs are not guaranteed to be truthful and that hallucination persists even with high-quality academic training data.

Reception and debate

The model card frames Galactica as a possible route into academic literature and a base for tools in fields such as mathematics, biology and chemistry. The authors presented examples where it stood in for ordinary search tools. They also urged researchers to look for useful applications while keeping hallucination and bias in mind.

Why it matters

The team argues that language models could become a fresh interface for science and expects a wave of scientific tools built on systems like this one. They released the model openly to support that direction and invited the research community to explore new uses.

Questions people ask about Galactica

What is Galactica for?

It targets scientific work: predicting citations, answering research questions, mathematical reasoning, summarising, drafting documents, estimating molecular properties and extracting entities. The team built it to explore whether a language model could help organise scientific knowledge rather than only search through it.

Who made it and when?

The Papers with Code team at Meta AI developed it. A paper describing the model was posted on arXiv on 16 November 2022, and the weights were released openly for the scientific community at the same time.

How big is it?

The family covers several sizes, from 125 million parameters up to 120 billion. Training used 106 billion tokens of open-access scientific material, including papers, textbooks, encyclopaedias and knowledge bases.

Can I use it commercially?

No. The weights come under a Creative Commons BY-NC 4.0 licence, which restricts use to non-commercial purposes. The model card also advises against production use without safeguards because the system can generate false content.

What are its main weaknesses?

It fabricates information, and academic training data does not stop this, particularly for less cited topics. Citation predictions lean towards popular work even at larger scales. Bias remains on some measures despite lower toxicity rates than other large models.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Galactica yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2211.09085, huggingface.co/facebook/galactica-120b. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.