News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM

Search: Transformer

Concepts

Switch Transformer WorldSwitch Transformer A sparse mixture-of-experts language model scaling to a trillion parameters.Research · Research only · 2021 fnGPT-5.6 🇺🇸 United StatesGPT-5.6 OpenAI's 2026 model family in Luna, Terra and Sol tiersGeneral · API only · 2026 XLNet WorldXLNet Permutation-based autoregressive transformer model created by Google AIResearch · Open weights · 2019 GPT-2 🇺🇸 United StatesGPT-2 OpenAI's 2019 text model that predicts the next word in a sequenceGeneral · Open weights · 2019 PaLM 🇺🇸 United StatesPaLM Google AI's 540-billion-parameter dense decoder-only transformer language modelGeneral · API only · 2022 ELMo 🇺🇸 United StatesELMo A 2018 bidirectional LSTM that produced context-sensitive word embeddings.Research · Research only · 2018 imgwavGPT-4 🇺🇸 United StatesGPT-4 Multimodal large language model by OpenAIGeneral · API only · 2023 BERT WorldBERT An encoder-only transformer that reads text in both directions at once.Research · Open weights · 2018 imgfnGPT-5.3-Codex 🇺🇸 United StatesGPT-5.3-Codex OpenAI's code-focused large language model with repository and terminal accessCode · API only · 2026 Chinchilla 🇬🇧 United KingdomChinchilla A 70-billion-parameter DeepMind model built to test how models should scale.Research · Research only · 2022 OpenAI o1 🇺🇸 United StatesOpenAI o1 OpenAI's first reasoning model, which thinks before answering.Reasoning · API only · 2024 GPT-1 WorldGPT-1 OpenAI's first GPT model, a 117M-parameter decoder-only transformer from 2018.Research · Open weights · 2018 fnGPT-5.4 WorldGPT-5.4 OpenAI's March 2026 large language model with computer use and deep research.General · API only · 2026 GPT-4chan WorldGPT-4chan A GPT-J model fine-tuned on posts from 4chan's /pol/ board.General · Open weights · 2022 DBRX WorldDBRX A 132-billion-parameter mixture-of-experts language model from Databricks.General · Conditional licence · 2024 GPT-3 🇺🇸 United StatesGPT-3 A large language model with 175 billion parameters released by OpenAI in 2020.General · API only · 2020 Minerva 🇮🇹 ItalyMinerva An Italian-first large language model trained from scratch at Sapienza University of Rome.General · Open weights · 2024 GPT-J WorldGPT-J A 6-billion-parameter open English text generator from EleutherAI, released in 2021.General · Open weights · 2021 wavWhisper WorldWhisper OpenAI's open-source model for turning speech into text and translating it.General · Open weights · 2022 T5 🇺🇸 United StatesT5 Encoder-decoder language model family from Google AI using a text-to-text framework.General · Fully open · 2019 wavMamba WorldMamba Efficient sequence modelling architecture using state spacesResearch · Open weights imgwavfnGato 🇬🇧 United KingdomGato A single DeepMind network handling dialogue, games and robot controlGeneral · 2022 imgOpenAI o4-mini 🇺🇸 United StatesOpenAI o4-mini Multimodal reasoning model by OpenAI, now retired.Reasoning · API only · 2025 GPT-NeoX-20B 🇺🇸 United StatesGPT-NeoX-20B Open 20 billion parameter autoregressive language model by EleutherAIGeneral · Open weights · 2022 fnSmolLM3 🇺🇸 United StatesSmolLM3 Open 3B parameter language model with reasoning modesGeneral · Open weights GPT-Neo WorldGPT-Neo Open autoregressive language model by EleutherAIGeneral · Open weights Qwen-72B 🇨🇳 ChinaQwen-72B A 72-billion parameter open-weight language model from Alibaba Cloud.General · Open weights · 2023 Pharia-1-LLM-7B 🇩🇪 GermanyPharia-1-LLM-7B Multilingual 7B parameter decoder model by Aleph Alpha ResearchGeneral · Open weights RuGPT-3 🇷🇺 RussiaRuGPT-3 Russian language model based on GPT-2 architectureRegional · Open weights Falcon-H1R-7B 🇦🇪 United Arab EmiratesFalcon-H1R-7B A 7-billion parameter reasoning model with hybrid architectureReasoning · Open weights · 2026 Falcon-H1 🇦🇪 United Arab EmiratesFalcon-H1 Hybrid transformer and state-space model series by TIIGeneral · Open weights · 2025 Krutrim-1 🇮🇳 IndiaKrutrim-1 Multilingual language model for Indian languagesGeneral · Conditional licence · 2024 Krutrim-2 🇮🇳 IndiaKrutrim-2 A 12-billion parameter multilingual language model focused on Indian languages and context.General · Open weights · 2024 Hunyuan-TurboS 🇨🇳 ChinaHunyuan-TurboS Hybrid Mamba-Transformer MoE model by TencentGeneral · API only · 2025 OPT 🇺🇸 United StatesOPT Meta AI's family of open decoder-only language models from 125M to 175B parametersGeneral · Conditional licence · 2022 Jamba2 Mini WorldJamba2 Mini A small open-weights SSM-Transformer model for enterprise workSmall · Open weights Mamba-2 🇺🇸 United StatesMamba-2 A state-space model architecture for efficient language modellingResearch · Fully open · 2024 AlphaCode 🇬🇧 United KingdomAlphaCode A code generation system that solves competitive programming problems.Code · Research only · 2022 Zamba2 🇺🇸 United StatesZamba2 Hybrid Mamba2-transformer language model suite by ZyphraGeneral · Open weights · 2024 Hunyuan-Large 🇨🇳 ChinaHunyuan-Large Open-source mixture of experts model with 389 billion parametersGeneral · Open weights · 2024 RETRO WorldRETRO A retrieval-enhanced transformer that conditions on chunks from a huge document database.Research · Research only · 2021 Falcon Mamba 7B 🇦🇪 United Arab EmiratesFalcon Mamba 7B A 7-billion parameter language model using the Mamba architecture.General · Open weights · 2024 fnNemotron 3 Nano 🇺🇸 United StatesNemotron 3 Nano NVIDIA's 30B hybrid Mamba-Transformer MoE model for reasoning and agentsGeneral · Conditional licence · 2025 PaLM 2 🇺🇸 United StatesPaLM 2 Google's multilingual Transformer language model in four sizes, from on-device to largestGeneral · API only · 2023 ModernBERT 🇺🇸 United States and 1 moreModernBERT A modernised encoder model for retrieval and classification tasks.Encoder · Open weights · 2024 RWKV-4 WorldRWKV-4 A dense recurrent model with transformer-level performanceGeneral · Open weights · 2023 fnGranite 4.0 🇺🇸 United StatesGranite 4.0 IBM's hybrid Mamba/transformer language models for enterprise workGeneral · Open weights · 2025 Phi-2 🇺🇸 United StatesPhi-2 A 2.7 billion parameter Microsoft research model for QA, chat and codeResearch · Open weights fnGpt-oss-120b 🇺🇸 United StatesGpt-oss-120b OpenAI's open-weight reasoning model that fits on one 80GB GPUReasoning · Open weights · 2025 Gemma 2 🇺🇸 United StatesGemma 2 Open lightweight language models from Google DeepMind, 2B to 27B parametersGeneral · Open weights · 2024 MT5 🇺🇸 United StatesMT5 A multilingual text-to-text transformer pretrained on 101 languagesGeneral · Open weights · 2020 Gopher 🇬🇧 United KingdomGopher A 280 billion parameter Transformer language model from DeepMind, studied for how scale changes ability.Research · Research only · 2021 BART 🇺🇸 United StatesBART A denoising encoder-decoder transformer for generation and comprehension tasksGeneral · Conditional licence · 2019 Phi-1 🇺🇸 United StatesPhi-1 A small Microsoft model specialised for basic Python codingCode · Open weights · 2023 Megatron-LM 🇺🇸 United StatesMegatron-LM Model-parallel training techniques for transformer language models with billions of parametersResearch · Fully open · 2019 Codestral Mamba 🇫🇷 FranceCodestral Mamba A 7B open code model built on the Mamba2 architectureCode · Open weights · 2024 Pythia 🇺🇸 United StatesPythia A suite of 16 models for interpretability research.Research · Open weights · 2023 BioBERT WorldBioBERT Biomedical language model based on BERT architectureDomain · Open weights · 2019 DialoGPT 🇺🇸 United StatesDialoGPT A GPT-style model trained on Reddit threads to write chat repliesGeneral · Open weights · 2019 fnNemotron Nano 2 🇺🇸 United StatesNemotron Nano 2 A 9B hybrid Mamba-Transformer model for reasoning and chat, from NVIDIA.General · Open weights · 2025

The weekly digest

Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.