Word2vec
A 2013 Google method for learning word embeddings from which much of modern language modelling grew.
Word2vec, published by Tomas Mikolov and colleagues at Google in 2013, trained a shallow network to predict a word from its neighbours, or the neighbours from the word. The by-product was a table of word vectors in which simple arithmetic captured relationships, the famous example being that king minus man plus woman lands near queen. Word2vec gave every word a single fixed vector, whatever its context. Later models such as ELMo and BERT made embeddings depend on the surrounding sentence, which handles words with several meanings.