All models
471 models so far, best known first. 21 more are on the list to be written.
WorldBERT
An encoder-only transformer that reads text in both directions at once.Research · Open weights · 2018
🇮🇳 IndiaMuRIL
A BERT-style encoder pre-trained on 17 Indian languages and their transliterationsEncoder · Open weights · 2021
🇺🇸 United StatesRoBERTa
A robustly optimised BERT pretraining study with released models and codeEncoder · Open weights · 2019
WorldBioBERT
Biomedical language model based on BERT architectureDomain · Open weights · 2019
🇺🇸 United StatesDeBERTa
A BERT-style encoder with disentangled attention and an enhanced mask decoder.Encoder · Open weights · 2020
🇮🇳 IndiaIndicBERT
Multilingual encoder model for Indian languagesEncoder · Open weights · 2022
🇩🇪 GermanySentence-BERT
Efficient sentence embeddings using Siamese BERT networksEncoder · Open weights · 2019
🇩🇪 GermanyJina Embeddings v3
Multilingual text embedding model with task-specific adaptersEncoder · Open weights · 2024
🇺🇸 United StatesNomic Embed
Open-source long-context text embedding modelEncoder · Open weights · 2024
WorldDistilBERT
A smaller, faster version of the BERT language model.Encoder · Open weights · 2019
🇺🇸 United StatesSciBERT
A BERT model trained on scientific text.Domain · Open weights · 2019
🇫🇷 FranceCamemBERT
French language model based on RoBERTa architectureEncoder · Open weights · 2019
WorldAfriBERTa
Multilingual encoder model for eleven African languagesEncoder · Open weights · 2021
🇺🇸 United StatesALBERT
A slimmed-down BERT encoder that shares weights across its layersEncoder · Open weights · 2019
🇺🇸 United States and 1 moreModernBERT
A modernised encoder model for retrieval and classification tasks.Encoder · Open weights · 2024