News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
GPT-3

GPT-3

A large language model with 175 billion parameters released by OpenAI in 2020.

OpenAI released GPT-3 in May 2020 as the third iteration of its generative pre-trained transformer series. With 175 billion parameters, it demonstrated strong zero-shot and few-shot learning capabilities across various tasks.

Share on WhatsApp

  • Released by OpenAI in May 2020 with 175 billion parameters.
  • Uses a decoder-only transformer architecture with attention mechanisms.
  • Microsoft secured an exclusive license for the underlying model.
  • Demonstrates strong zero-shot and few-shot learning capabilities.
  • Trained on hundreds of billions of words from multiple sources.

What it is

GPT-3 is a large language model developed by OpenAI and launched in May 2020. It belongs to the GPT series and serves as a decoder-only transformer architecture. The model contains 175 billion parameters, making it significantly larger than previous versions like GPT-2. It processes text using a context window of 2,048 tokens. Its primary function is generating human-like text based on input prompts. It supports zero-shot and few-shot learning for diverse tasks.

How it came about

OpenAI published the research paper for GPT-3 on May 28, 2020. A team of 31 researchers detailed the development process. Microsoft announced an exclusive licensing deal for the model on September 22, 2020. While Microsoft gained access to the underlying model, the public API remained available to others. OpenAI opened the API for beta testing on June 11, 2020. Later updates included InstructGPT and the GPT-3.5 series.

Inside the model

The model uses a transformer architecture that relies on attention mechanisms to focus on relevant text segments. It was trained on hundreds of billions of words. The dataset included filtered Common Crawl data, WebText2, Books1, Books2, and Wikipedia. Training required significant computational resources. The system does not need further training for distinct language tasks due to its broad pre-training data. It employs byte-pair encoding for tokenization.

What it can do

GPT-3 excels at zero-shot and few-shot learning across many tasks. It can generate code in languages such as Python, CSS, and JSX. Human evaluators found it difficult to distinguish its articles from human writing. However, it occasionally produces toxic language similar to its training data. Studies showed it performed slightly better than random guessing in distinguishing itself from humans. It lacks deep understanding of word meanings despite coherent output.

How to use it

Initially, OpenAI offered access through a public API starting in June 2020. Microsoft holds an exclusive license for the underlying model. Other users access the model via the API. OpenAI restricted API access temporarily before implementing safeguards. By November 2021, unrestricted access was restored with content moderation tools. Specific model sizes like davinci are accessible through the API.

Safety and testing

The model generates toxic language due to mimicking its training data. OpenAI implemented strategies to limit this toxicity. A University of Washington study compared its toxicity levels to other models. Researchers highlighted risks like misinformation, spam, and phishing. OpenAI warned against using the model for medical advice. One test resulted in harmful advice being given to a simulated patient.

Reception and debate

Reactions were mixed, ranging from awe to concern. Critics noted the model's lack of true comprehension. Some described the output as spooky or terrifying. Others pointed out safety issues regarding biased language. Philosophers and AI experts debated its scientific value. Sam Altman criticized the hype surrounding the model. Despite limitations, it was seen as an impressive step in language processing.

Questions people ask about GPT-3

What is GPT-3?

GPT-3 is a large language model released by OpenAI in May 2020. It uses a transformer architecture with 175 billion parameters to generate text. It supports zero-shot and few-shot learning for various tasks.

Who owns GPT-3?

OpenAI developed GPT-3. Microsoft has an exclusive license for the underlying model. The public API remains available to other users under specific terms set by OpenAI.

How large is GPT-3?

GPT-3 has 175 billion parameters. Each parameter uses 16-bit precision. This requires approximately 350GB of storage space for the model weights.

Can GPT-3 write code?

Yes, GPT-3 can generate code in languages like Python, CSS, and JSX. It was used to create GitHub Copilot. The Codex variant is specifically designed for code generation tasks.

Good reading and sources

Where this page comes from

The facts on this page come from the Wikipedia article "GPT-3", written by its contributors and used under the CC BY-SA 4.0 licence, and from its Wikidata record (CC0). EveryLLM rewrote the text in its own words, with AI assistance, and compares every page with the article to make sure no passage is copied. It may be reused under the same CC BY-SA 4.0 licence with a link back.

It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-16.

The weekly digest

Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.