Release anniversaries, 11 Jun 2027
GPT-1 release
On 11 June 2018 OpenAI described GPT-1, the first generative pre-trained transformer.
On 11 June 2018 OpenAI published "Improving Language Understanding by Generative Pre-Training", by Alec Radford and colleagues. It described a 117-million-parameter decoder-only transformer pre-trained on a collection of about 7,000 unpublished books and then fine-tuned for individual tasks such as question answering and judging whether one sentence follows from another. The model improved results on most of the tasks tested. Its recipe, pre-train on plain text and then adapt, named the GPT series and set the pattern for the models that followed.