News
ResearchOpenAI pens AI safety agreement with four firmsReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierOpenAI鈥檚 New Hacking Model Found Dozens Of Holes In Apple TechFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIResearchOpenAI pens AI safety agreement with four firmsReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierOpenAI鈥檚 New Hacking Model Found Dozens Of Holes In Apple TechFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAI
EveryLLM

Speech and audio

Listens and speaks. 41 models on EveryLLM so far.

Speech and audio models listen and speak. Older voice assistants chained three systems: speech recognition to turn speech into text, a language model, and text-to-speech. Newer models are trained on audio directly, so they hear tone and hesitation, can reply in natural speech within a fraction of a second, and let users interrupt them mid-sentence.

Listening

OpenAI's Whisper, released as open source in September 2022, was trained on 680,000 hours of multilingual audio from the web and became a standard tool for transcription. Language models now accept audio directly: Gemini models take long recordings, and Alibaba's Qwen2.5-Omni (March 2025) reads text, images, audio and video in one model. In India, Sarvam AI and the government's Bhashini programme focus on speech in Indian languages.

Speaking

GPT-4o, shown in May 2024, answered spoken questions in about 320 milliseconds on average, close to human turn-taking, and ChatGPT's Advanced Voice Mode reached paying users from September 2024. In July 2024 the Paris lab Kyutai demonstrated Moshi, a model that listens and speaks at the same time, and later released it openly. Voice cloning raises fraud risks, so most providers limit the choice of voices.

Famous speech and audio models

Still to be written: GPT-4o.

Further reading

imgwavvidGemini 馃嚭馃嚫 United StatesGemini Google DeepMind's multimodal LLM family, from Nano to UltraGeneral 路 API only 路 2023 imgwavvidGemma 馃嚭馃嚫 United StatesGemma Google DeepMind's open family of Gemini-derived language modelsGeneral 路 Open weights 路 2024 imgwavInkling WorldInkling An open-weights mixture-of-experts model handling text, images and audio inputGeneral 路 Open weights 路 2026 imgwavvidQwen 馃嚚馃嚦 ChinaQwen A family of large and small language models developed by Alibaba Cloud.General 路 Open weights 路 2023 imgwavKimi 馃嚚馃嚦 ChinaKimi A Chinese chatbot and large language model family by Moonshot AI.Assistant 路 API only 路 2023 imgwavvidNemotron 馃嚭馃嚫 United StatesNemotron Nvidia family of language and multimodal models for enterprise and reasoningGeneral 路 Conditional licence 路 2023 imgwavvidMiMo-V2.6-Pro 馃嚚馃嚦 ChinaMiMo-V2.6-Pro Xiaomi鈥檚 1T parameter multimodal model with 1M token contextMultimodal 路 Open weights imgwavvidGemini 1.5 Pro 馃嚭馃嚫 United StatesGemini 1.5 Pro Google DeepMind's mid-size multimodal model with up to a 1 million token context windowGeneral 路 API only 路 2024 imgwavvidGemma 4 馃嚭馃嚫 United StatesGemma 4 Google DeepMind's open multimodal models in five sizes, from phones to serversGeneral 路 Open weights imgwavvidQwen3-Omni 馃嚚馃嚦 ChinaQwen3-Omni Multimodal model processing text, audio, video and images with real-time speech output.Multimodal 路 Open weights 路 2025

The weekly digest

Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.