BERT
Google's 2018 encoder model, which read text in both directions and set new records across language tasks.
Also: Bidirectional Encoder Representations from Transformers
BERT was released by Jacob Devlin and colleagues at Google in October 2018. It was an encoder-only transformer pre-trained with masked language modelling and next-sentence prediction on English Wikipedia and a large collection of books, in sizes of 110 million and 340 million parameters. Fine-tuned for a task, it topped benchmarks such as GLUE and SQuAD. Google began using it in Search in 2019. BERT inspired a large family, including RoBERTa, ALBERT, DistilBERT and Indian-language models such as IndicBERT and MuRIL.