News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM

Text

Reads and writes text. 471 models on EveryLLM so far.

Text is the base modality of every language model: it reads a prompt and writes a reply, token by token. A tokenizer first splits text into tokens, which are whole words, pieces of words or single characters; in English a token is roughly three-quarters of a word. Everything else a model does, from code to reasoning to calling tools, is ultimately expressed as text that it reads or writes.

Tokens and context

Most models use byte-pair encoding, which builds a vocabulary of frequent character sequences. Vocabularies have grown from about 50,000 tokens in GPT-2 to around 200,000 in GPT-4o and 256,000 in Gemma. The context window is how many tokens the model can consider at once: GPT-3 handled 2,048 in 2020, while Gemini 1.5 Pro offered one million in 2024. Languages in non-Latin scripts often need more tokens for the same meaning.

What text-only still means

Many models remain text-only, especially smaller ones and older families such as GPT-3, Llama 2 and BERT, and text still carries most business uses: drafting, summarising, pulling fields out of documents, translation and search. Text-only models are cheaper to train and serve, and easier to evaluate, since their inputs and outputs can be checked and compared directly.

Famous text models

Further reading

ELMo 🇺🇸 United StatesELMo A 2018 bidirectional LSTM that produced context-sensitive word embeddings.Research · Research only · 2018 wavMamba WorldMamba Efficient sequence modelling architecture using state spacesResearch · Open weights Evo 🇺🇸 United StatesEvo Open genomic language models that read and write DNA, RNA and protein sequencesDomain · Fully open · 2024 Codestral Mamba 🇫🇷 FranceCodestral Mamba A 7B open code model built on the Mamba2 architectureCode · Open weights · 2024 RWKV-7 Goose WorldRWKV-7 Goose Recurrent language model with constant memory usageGeneral · Open weights · 2025 Zamba WorldZamba A 7-billion parameter hybrid language model combining Mamba and transformer layers.General · Open weights · 2024 Eagle and Finch WorldEagle and Finch Efficient recurrent sequence models with matrix statesGeneral · Fully open · 2024 Falcon-H1 🇦🇪 United Arab EmiratesFalcon-H1 Hybrid transformer and state-space model series by TIIGeneral · Open weights · 2025 Mamba-2 🇺🇸 United StatesMamba-2 A state-space model architecture for efficient language modellingResearch · Fully open · 2024 Zamba2 🇺🇸 United StatesZamba2 Hybrid Mamba2-transformer language model suite by ZyphraGeneral · Open weights · 2024 Falcon Mamba 7B 🇦🇪 United Arab EmiratesFalcon Mamba 7B A 7-billion parameter language model using the Mamba architecture.General · Open weights · 2024 RWKV-4 WorldRWKV-4 A dense recurrent model with transformer-level performanceGeneral · Open weights · 2023

The weekly digest

Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.