News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
Mamba-2

Mamba-2

A state-space model architecture for efficient language modelling

Mamba-2 is a deep learning architecture introduced in May 2024 by researchers Tri Dao and Albert Gu. It refines the earlier Mamba design using structured state space duality to achieve faster processing speeds while maintaining performance comparable to Transformer models on language tasks.

Share on WhatsApp

  • Released in May 2024 by Tri Dao and Albert Gu
  • Uses structured state space duality framework
  • Claims to be 2-8 times faster than original Mamba
  • Competitive with Transformers on language modelling
  • Based on structured semiseparable matrix decompositions

What it is

Mamba-2 is a neural network architecture designed for language modelling. It was presented in a research paper published in May 2024. The model builds upon the earlier Mamba architecture, which uses state-space models instead of the standard attention mechanisms found in Transformers. Mamba-2 aims to provide a more efficient computational structure while retaining competitive performance on language tasks. It represents a theoretical bridge between state-space models and attention-based variants.

How it came about

The architecture was developed by Tri Dao and Albert Gu. Their work was published on arXiv in May 2024 under the title Transformers are SSMs. This release followed the introduction of the original Mamba model, which had demonstrated that state-space models could match or exceed Transformer performance at small to medium scales. The new version seeks to refine these gains through improved algorithmic design.

Inside the model

The core innovation in Mamba-2 is the application of structured state space duality. This framework establishes theoretical connections between state-space models and various forms of attention mechanisms. These links are derived from decompositions of structured semiseparable matrices. The resulting architecture refines the selective state-space layer used in the original Mamba. This design choice allows the model to process data significantly faster than its predecessor.

What it can do

The developers claim that Mamba-2 is two to eight times faster than the original Mamba architecture. Despite this increase in speed, the model remains competitive with Transformer models on language modelling benchmarks. The paper suggests that the family of state-space models and Transformers are closely related, allowing Mamba-2 to leverage efficiencies without sacrificing predictive accuracy on standard language tasks.

How to use it

The research paper and associated code are available through open platforms such as arXiv and Hugging Face. The work is presented as part of the broader open science community, with tools like CatalyzeX and Replicate listed for code discovery and execution. There are no indications of proprietary restrictions on the architectural concepts described in the publication.

Why it matters

Mamba-2 contributes to the growing body of research challenging the dominance of Transformer architectures. By demonstrating that state-space models can be both faster and competitive, it offers an alternative path for scaling deep learning systems. The theoretical framework of structured state space duality may influence future designs of efficient sequence models.

Questions people ask about Mamba-2

What is Mamba-2?

Mamba-2 is a deep learning architecture for language modelling that refines the original Mamba model. It uses structured state space duality to improve efficiency while maintaining performance levels similar to Transformer models.

How does Mamba-2 differ from Transformers?

While Transformers rely on attention mechanisms, Mamba-2 is based on state-space models. The authors show these approaches are closely related through matrix decompositions, but Mamba-2 offers significant speed improvements over previous state-space designs.

Who developed Mamba-2?

The model was developed by researchers Tri Dao and Albert Gu. They published their findings in a paper titled Transformers are SSMs on arXiv in May 2024, detailing the theoretical connections and architectural refinements.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Mamba-2 yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2405.21060. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.