All models
471 models so far, best known first. 21 more are on the list to be written.
WorldELECTRA
A pre-training method that trains text encoders to spot replaced tokens.Research · Research only · 2020
WorldRETRO
A retrieval-enhanced transformer that conditions on chunks from a huge document database.Research · Research only · 2021
🇺🇸 United StatesToolformer
A language model that learns on its own when to call external tools through APIs.Research · Research only · 2023
WorldPaLM-E
An embodied multimodal language model that grounds words in robot sensor inputMultimodal · Research only · 2023
🇺🇸 United StatesGalactica
A language model trained on scientific text for research tasksDomain · Conditional licence · 2022
WorldMM1
A family of multimodal language models up to 30B parameters, dense and MoE.Multimodal · Research only · 2024
🇺🇸 United StatesAlexaTM 20B
A 20-billion-parameter multilingual sequence-to-sequence model from Amazon's Alexa team.General · Research only · 2022
🇺🇸 United StatesOrca 2
A small LLaMA-2-based model trained on synthetic data to reason.Small · Conditional licence · 2023
🇺🇸 United StatesPhi-2
A 2.7 billion parameter Microsoft research model for QA, chat and codeResearch · Open weights
🇺🇸 United StatesInstructGPT
GPT-3 models fine-tuned with human feedback to follow instructionsGeneral · API only · 2022
🇺🇸 United StatesWebGPT
A GPT-3 model fine-tuned to answer questions while browsing the webGeneral · Research only · 2021
🇺🇸 United StatesRecurrentGemma
Open language models using Google's Griffin recurrent-plus-attention architectureGeneral · Open weights · 2024
🇺🇸 United StatesMed-PaLM
A medical-domain language model aligned from Flan-PaLM using instruction prompt tuning.Domain · Research only · 2022
WorldSwitch Transformer
A sparse mixture-of-experts language model scaling to a trillion parameters.Research · Research only · 2021
🇺🇸 United StatesGLaM
A trillion-parameter mixture-of-experts language model from Google ResearchResearch · Research only · 2021
🇺🇸 United StatesMinerva
A language model trained further on technical content to solve maths and science problemsResearch · Research only · 2022
🇺🇸 United StatesALBERT
A slimmed-down BERT encoder that shares weights across its layersEncoder · Open weights · 2019
🇬🇧 United KingdomFlamingo
A visual language model family that learns new image and video tasks from a few examples.Multimodal · Research only · 2022
🇬🇧 United KingdomGopher
A 280 billion parameter Transformer language model from DeepMind, studied for how scale changes ability.Research · Research only · 2021