News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
wavMoshi

Moshi

Real-time full-duplex spoken dialogue model

Moshi is a speech-text foundation model for real-time spoken dialogue, released in September 2024 by Kyutai. It processes audio and text in parallel streams to enable low-latency, full-duplex conversation without explicit speaker turns.

Share on WhatsApp

  • First real-time full-duplex spoken LLM
  • Practical latency of 200 milliseconds
  • Uses Inner Monologue for text-audio alignment
  • Trained on 7 million hours of audio
  • Parallel streams model overlapping speech

What it is

Moshi is a foundation model designed for spoken dialogue that handles both speech and text. Developed by Kyutai and announced in September 2024, it aims to replicate natural conversation dynamics. Unlike traditional systems that process voice activity, recognition, and synthesis separately, Moshi generates speech directly from audio tokens. This approach allows for full-duplex interaction, meaning both parties can speak simultaneously. The model supports real-time exchange with practical latency around 200 milliseconds. It is built primarily for casual conversation and basic advice rather than complex tool use.

How it came about

The model was introduced in a paper submitted to arXiv in September 2024 by researchers including Alexandre Défossez and Neil Zeghidour. Kyutai developed the system to address limitations in pipeline-based spoken dialogue frameworks. Previous methods suffered from high latency and loss of non-linguistic cues like emotion. Moshi emerged as a solution to model overlapping speech and interruptions naturally. The team released weights and code via Hugging Face and GitHub shortly after the paper publication.

Inside the model

Moshi uses a text language model backbone extended to handle audio. It generates speech tokens using a neural audio codec called Mimi, which operates at 12Hz. The architecture features parallel streams for user and model speech, removing the need for turn-taking segmentation. A technique called Inner Monologue predicts time-aligned text tokens before audio tokens. This method improves linguistic quality and enables streaming recognition. Training involved 1016 Nvidia H100 GPUs. Data included millions of hours of English audio and filtered web text.

What it can do

The model supports real-time full-duplex dialogue with theoretical latency of 160ms. It handles overlapping speech, interruptions, and interjections better than pipeline systems. Moshi provides streaming speech recognition and text-to-speech capabilities. Its conversational skills cover casual chat, trivia, and roleplay. However, it cannot access external tools or perform complex professional tasks. The system produces only one voice to prevent impersonation. Linguistic quality is enhanced by the Inner Monologue prefix mechanism.

How to use it

Kyutai offers Moshi weights under a research-only licence. The model is available on Hugging Face in PyTorch format quantized to bf16 precision. Code resides in a public GitHub repository. A live demo is accessible via the Moshi website. The developers advise against using it for professional duties or malicious purposes. Components like the Mimi codec can be repurposed for other speech tasks with fine-tuning.

Safety and testing

Training included safeguards to limit toxic outputs. Analysis shows toxicity levels comparable to existing models. Bias exists towards topics over-represented in training data. The system restricts output to a single voice to avoid impersonation risks. Developers note that sociotechnical limitations require further study over time. Malicious use and impersonation are explicitly out of scope for intended usage.

Why it matters

Moshi demonstrates that speech-to-speech generation can reduce latency in spoken dialogue. It challenges the standard pipeline of separate recognition and synthesis modules. The Inner Monologue approach offers a new method for aligning text and audio tokens. The open release of weights and the Mimi codec supports further research in speech language models. It highlights the potential for modeling arbitrary conversational dynamics in real time.

Questions people ask about Moshi

What makes Moshi different from other voice assistants?

Moshi processes speech directly without converting to text first. It allows both users to speak at the same time, known as full-duplex. This reduces latency to about 200ms and preserves emotional cues lost in text.

Can Moshi understand multiple languages?

No, the model card specifies that Moshi is trained primarily on English data. The unsupervised audio dataset consists mostly of English speech. It is not designed for multilingual dialogue at this stage.

Is Moshi available for commercial use?

No, the model is released for research purposes only. The developers do not recommend it for professional duties. Weights are available on Hugging Face but under a licence restricting commercial deployment.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Moshi yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2410.00037, huggingface.co/kyutai/moshiko-pytorch-bf16. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.