Release anniversaries, 12 Jun 2027
"Attention Is All You Need" published
On 12 June 2017 eight Google researchers posted the paper that introduced the transformer.
The paper "Attention Is All You Need" was posted to the arXiv preprint server on 12 June 2017 by Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Łukasz Kaiser and Illia Polosukhin, then at Google Brain and Google Research. It proposed the transformer, a network that handled translation using attention alone, without recurrence, and trained much faster on graphics processors. It was presented at NIPS that December. Within two years it underpinned GPT, BERT and T5, and nearly every language model since has been a transformer.