Small: under 1 billion
BERT, GPT-2 small and the tiny models that run on a phone or a laptop. 6 models.
WorldXLNet
Permutation-based autoregressive transformer model created by Google AIResearch · Open weights · 2019
WorldGPT-1
OpenAI's first GPT model, a 117M-parameter decoder-only transformer from 2018.Research · Open weights · 2018
🇺🇸 United States and 1 moreModernBERT
A modernised encoder model for retrieval and classification tasks.Encoder · Open weights · 2024