PaLM
Google AI's 540-billion-parameter dense decoder-only transformer language model
PaLM, short for Pathways Language Model, is a dense decoder-only transformer built by Google AI and announced in April 2022. Its largest version holds 540 billion parameters, and it matters because it showed strong reasoning and code skills at that scale and spawned medical, vision and speech variants.
- Largest PaLM holds 540 billion parameters; 8B and 62B builds tested scaling
- Trained on 780 billion tokens, half of it social media conversation
- Ran on 6,144 TPU v4 chips across two Pods, hitting 57.8% hardware FLOPs utilisation
- Med-PaLM was first to pass U.S. medical licensing questions
- Announced April 2022; API opened March 2023
What it is
PaLM stands for Pathways Language Model. Google AI produced it as a large language model built on a dense decoder-only transformer design, meaning every one of its parameters is active when it processes text. The headline version carries 540 billion parameters. Two smaller builds, at 8 billion and 62 billion, were trained alongside it so researchers could compare what changes as models grow. It handles commonsense and arithmetic reasoning, joke explanation, code writing and translation. Announced in April 2022, it sat inside Google until an API opened in March 2023.
How it came about
The model surfaced publicly in April 2022 but stayed closed to outside users for close to a year. Access arrived in March 2023 through an API covering PaLM and other Google technologies, first for developers who joined a waiting list and then for everyone. Google and DeepMind adapted the 540-billion-parameter build into Med-PaLM, tuned on medical data. Google also paired PaLM with a vision transformer to make PaLM-E. PaLM 2 was unveiled at the Google I/O keynote in May 2023, and AudioPaLM followed in June 2023.
Inside the model
Pre-training ran on a curated corpus of 780 billion tokens spanning many natural-language tasks. The mix drew on cleaned web pages, books, Wikipedia entries, news stories, source code taken from public GitHub repositories and social media discussion, building on the data recipe behind LaMDA. Half of the corpus is social media conversation, which supports dialogue ability. The 540-billion-parameter run used two TPU v4 Pods, each with 3,072 TPU v4 chips linked to 768 hosts, combining model and data parallelism. That 6,144-chip arrangement was the largest TPU setup used and reached 57.8% hardware FLOPs utilisation.
What it can do
PaLM covers commonsense reasoning, arithmetic reasoning, joke explanation, code generation and translation. Adding chain-of-thought prompting, where the model works through intermediate steps, brought sizeable gains on multi-step material such as word problems and logic puzzles. Med-PaLM, the medical fine-tune, beat earlier systems on medical question-answering benchmarks and was the first to reach a passing level on U.S. medical licensing questions. It answers multiple-choice and open-ended items, shows its reasoning and can judge its own replies.
How to use it
Outside access came through an API that Google launched in March 2023 alongside other technologies. A waiting list gated the first wave of developers before the service opened more widely. The model had been private from its April 2022 announcement until that point. No public weights or licence terms are given here.
Why it matters
PaLM became a base for several follow-on systems. Med-PaLM applied it to medicine, PaLM-E added a vision transformer so a robot could be directed without retraining, and AudioPaLM reused the PaLM 2 design and initialisation for speech-to-speech translation. PaLM 2, revealed in May 2023, is reported at 340 billion parameters trained on 3.6 trillion tokens. Within the Google line, PaLM took over from LaMDA and was later succeeded by Gemini.
Questions people ask about PaLM
What does PaLM stand for and who built it?
The name expands to Pathways Language Model. Google AI developed it, announcing the model in April 2022. It is a dense decoder-only transformer, so all of its parameters are used on every token it processes.
How big is PaLM?
The main version carries 540 billion parameters. Google researchers also trained 8-billion and 62-billion-parameter copies so they could study how behaviour shifts with scale. PaLM 2, the later model in the line, is reported at 340 billion parameters.
What data was it trained on?
Pre-training used 780 billion tokens drawn from filtered web pages, books, Wikipedia articles, news, source code from public GitHub repositories and social media chats. The recipe follows the one used for LaMDA, and social media conversation accounts for half of the total.
How can I use PaLM?
Google opened an API in March 2023, after the model had been private for roughly a year. Developers joined a waiting list at first, and the service later became publicly available. No downloadable weights are described.
What came out of PaLM?
Several variants grew from it. Med-PaLM is a medical fine-tune built with DeepMind, PaLM-E attaches a vision transformer for robotic control, and AudioPaLM handles speech-to-speech translation using the PaLM 2 architecture and initialisation.
Good reading and sources
- Official websiteOfficial site
- Google opens up its AI language model PaLM to challenge OpenAI and GPT-3The Verge, Article
- PaLM: Scaling Language Modeling with PathwaysarXiv, Article
- Google launches PaLM 2, its next-gen large language modelTechCrunch, Article
The facts on this page come from the Wikipedia article "PaLM", written by its contributors and used under the CC BY-SA 4.0 licence, and from its Wikidata record (CC0). EveryLLM rewrote the text in its own words, with AI assistance, and compares every page with the article to make sure no passage is copied. It may be reused under the same CC BY-SA 4.0 licence with a link back.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.