News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
wavWhisper

Whisper

OpenAI's open-source model for turning speech into text and translating it.

Whisper is a speech recognition and transcription model released by OpenAI in September 2022 under an MIT licence. Built as an encoder-decoder transformer trained on 680,000 hours of labelled audio, it handles many languages and translation into English, and has been widely adopted for transcription work across research, libraries and industry.

Share on WhatsApp

  • Released 21 September 2022 under the MIT License
  • Trained on 680,000 hours of labelled audio
  • Handles 96 non-English languages plus translation into English
  • Cuts errors by 55.2% versus other models across many datasets
  • Large V2 (Dec 2022) and Large V3 (Nov 2023) followed the first release

What it is

Whisper is a neural network that turns spoken audio into written text. OpenAI published it in September 2022 and made the code and weights freely available. It transcribes English and many other languages, and it can also render several non-English languages into English. The system is a deep learning acoustic model trained with weak supervision, meaning it learned from large amounts of automatically labelled data rather than carefully hand-checked examples. OpenAI says its mix of training data and post-training filtering gives it better handling of accents, background noise and specialist vocabulary than earlier methods. It is used for general sound understanding across many industries.

How it came about

Speech recognition moved from statistical techniques such as dynamic time warping and hidden Markov models to deep neural networks in the 2010s, helped by large datasets and faster hardware. Transformers, introduced by Google in 2017, then became the standard building block. A New York Times report said OpenAI ran out of high-quality text for its language models in 2021 and turned to YouTube and podcast transcripts, building Whisper to handle that. Large V2 arrived on 8 December 2022 and Large V3 in November 2023 at OpenAI Dev Day. In March 2025 OpenAI released GPT-4o and GPT-4o mini transcription models with lower error rates.

Inside the model

Audio is resampled to 16 kHz and turned into a log-magnitude mel spectrogram with 80 channels, then normalised. The encoder runs this through two convolutional layers, adds sinusoidal positional embeddings, and processes it with transformer blocks using pre-activation residual connections before layer normalisation. The decoder is a standard transformer of the same width, with learned positions and shared input-output embedding weights. It uses a byte-pair encoding tokenizer like GPT-2's; English-only versions reuse the GPT-2 vocabulary while multilingual ones use a retrained one of the same size. Special tokens mark language, task, timestamps and silence. Training used 680,000 hours of internet-sourced audio, including 117,000 hours across 96 non-English languages and 125,000 hours of translation data, with AdamW optimisation over 1 million updates.

What it can do

Whisper does not beat models specialised on the LibriSpeech benchmark, but across many datasets it is more robust and cuts errors by 55.2% compared with other models. Accuracy varies by language, with higher word error rates where training data is thin. The authors found multi-task training helped overall performance and believed the best model still underfits the data, so bigger models and longer training should improve results. Third-party studies found hallucinations: eight of ten public meeting transcripts, about half of 100 hours checked by an engineer, and nearly all of 26,000 transcripts reviewed by a developer. One study of 13,140 short clips found 187 hallucinations (1.4%), 38% of which inserted harmful false content.

How to use it

Whisper is published as open-source software under the MIT License, with code hosted on GitHub at github.com/openai/whisper. The repository is written in Python. Because the weights and code are freely downloadable, developers can run the model themselves or build it into their own products without a licensing fee.

Safety and testing

After the first model was trained, researchers ran it over subsets of the training data, ranked sources by error rate and size, and manually inspected the worst offenders to spot low-quality material such as partial transcripts or poor alignment. The model was then fine-tuned to suppress speaker names, and bad sources were dropped. Independent work has documented hallucinated output, including invented references to race, medications and violent events that were absent from the audio.

Reception and debate

Whisper is valued for general sound recognition and has been taken up across many industries. Emory University Libraries reported in a 2025 case study that it cut transcription labour by around 30-35%, moving staff effort from writing text to correcting it, though people still check accuracy, formatting and accessibility. OVALYTICS used it to transcribe YouTube videos and improve detection of offensive content. Researchers have also applied it in biomedical work, including a 2025 Alzheimer's disease study that transcribed spontaneous speech for classification alongside language model embeddings.

Why it matters

Whisper became a base for many speech and sound applications, from unified speech recognition to broader audio understanding. It is embedded in biomedical research workflows, content moderation systems, and the digitisation work of academic libraries and cultural heritage institutions. Its open release let others build on it directly, and its successors in OpenAI's transcription line, GPT-4o and GPT-4o mini, continue the same task with lower error rates.

Questions people ask about Whisper

What does Whisper actually do?

It takes spoken audio and writes it out as text. It works in English and many other languages, and it can also translate speech in several non-English languages into English. OpenAI built it as a deep learning acoustic model and released it as open-source software in September 2022.

How is Whisper trained?

It learned from 680,000 hours of audio paired with transcripts gathered from the internet, including 117,000 hours across 96 non-English languages and 125,000 hours of translation material. Audio was cut into 30-second clips, transcripts were cleaned and deduplicated, and the model ran for 1 million training updates.

Is Whisper free to use?

Yes. OpenAI published the code and weights under the MIT License, hosted on GitHub in Python. Anyone can download and run it, or build it into their own software, without paying a licence fee.

How accurate is it?

Across many datasets it makes 55.2% fewer errors than other models, though it does not beat specialists on LibriSpeech. Accuracy differs by language, and independent studies have found hallucinated output, including invented details not present in the audio.

What came after Whisper?

OpenAI released Large V2 on 8 December 2022 and Large V3 in November 2023. Then in March 2025 it brought out transcription models based on GPT-4o and GPT-4o mini, which have lower error rates than Whisper.

Good reading and sources

Where this page comes from

The facts on this page come from the Wikipedia article "Whisper (speech recognition system)", written by its contributors and used under the CC BY-SA 4.0 licence, and from its Wikidata record (CC0). EveryLLM rewrote the text in its own words, with AI assistance, and compares every page with the article to make sure no passage is copied. It may be reused under the same CC BY-SA 4.0 licence with a link back.

It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

The weekly digest

Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.