Search: glue
🇺🇸 United StatesALBERT
A slimmed-down BERT encoder that shares weights across its layersEncoder · Open weights · 2019
🇺🇸 United StatesRoBERTa
A robustly optimised BERT pretraining study with released models and codeEncoder · Open weights · 2019
🇺🇸 United StatesDeBERTa
A BERT-style encoder with disentangled attention and an enhanced mask decoder.Encoder · Open weights · 2020
WorldELECTRA
A pre-training method that trains text encoders to spot replaced tokens.Research · Research only · 2020