Compact: 1 to 10 billion
Phi, Gemma, Llama 3 8B, Mistral 7B and most on-device and edge models. 82 models.
🇺🇸 United StatesMegatron-LM
Model-parallel training techniques for transformer language models with billions of parametersResearch · Fully open · 2019
WorldEagle and Finch
Efficient recurrent sequence models with matrix statesGeneral · Fully open · 2024
🇮🇳 IndiaSarvam-Translate
Document-level translation model for 22 Indian languages.Domain · Fully open
🇺🇸 United StatesOLMoE
Open mixture-of-experts language model with 1 billion active parametersGeneral · Fully open · 2024