Transformer
The neural network design, introduced in 2017, that almost every modern language model is built on.
Also: transformer architecture
The transformer was described by eight Google researchers in the 2017 paper "Attention Is All You Need". It processes all the tokens of a text at once and uses attention to let each token gather information from the others, instead of reading word by word as recurrent networks did. A transformer is a stack of identical blocks, each with an attention step and a feed-forward step, joined by residual connections and normalisation. Because the work runs in parallel, transformers train efficiently on graphics processors, which made very large models possible.