Codestral Mamba
A 7B open code model built on the Mamba2 architecture
Codestral Mamba is a 7-billion-parameter instruction-tuned code model from Mistral AI, released in 2024 under Apache 2.0. Built on the Mamba2 architecture with input from Albert Gu and Tri Dao, it targets fast code assistance and long-sequence retrieval, and Mistral AI says it matches transformer code models of similar size.
- 7,285,403,648 parameters, instruction-tuned
- Built on the Mamba2 state-space architecture
- In-context retrieval tested up to 256k tokens
- Weights released under Apache 2.0
- Designed with help from Albert Gu and Tri Dao
What it is
Codestral Mamba is an instruction-tuned model for writing and reasoning about code, put out by Mistral AI in 2024. It carries 7,285,403,648 parameters and is aimed at programmers who want a fast assistant that can run on their own machine. Rather than the transformer design used by most language models, it follows the Mamba2 state-space approach, which Mistral AI says keeps response times quick regardless of how long the input is. The company positions it as a companion to its larger Codestral 22B model and as a step in its work on alternative architectures after the Mixtral family.
How it came about
The release followed Mistral AI's work on the Mixtral family and formed part of its programme of exploring new model designs. Albert Gu and Tri Dao, who are associated with the Mamba line of research, helped with the design. Mistral AI published the weights openly and said it hoped the model would encourage further study of architectures. It was made available on the company's platform under the identifier codestral-mamba-2407, next to Codestral 22B.
Inside the model
Instead of attention layers, the model uses the Mamba2 state-space design. Mistral AI says this gives linear-time inference, meaning the cost of generating each token does not grow with sequence length, and in theory allows sequences of unlimited length. The company reports testing in-context retrieval at lengths up to 256k tokens. Training focused on code and reasoning skills. The model is instruction-tuned and holds 7,285,403,648 parameters. Deployment tooling draws on the reference Mamba implementations, and the weights ship with a tokenizer file named tokenizer.model.v3.
What it can do
Mistral AI claims performance on a level with leading transformer code models of comparable size. Its published evaluation table reports 75.0% on HumanEval, 68.5% on MBPP, 58.8% on Spider, 57.8% on CruxE, 59.8% on HumanEval C++, 57.0% on HumanEval Java, 61.5% on HumanEval JS and 31.1% on HumanEval Bash. In that same table the larger Codestral 22B records higher figures on most of these tests, while DeepSeek v1.5 7B leads on several.
How to use it
The weights are downloadable from Hugging Face under Apache 2.0, allowing free use, changes and redistribution. Mistral AI recommends the mistral-inference SDK, and the model can also be served through TensorRT-LLM, with llama.cpp support flagged as forthcoming. A hosted version runs on la Plateforme as codestral-mamba-2407. By contrast, Codestral 22B needs a commercial licence for self-hosting or a community licence for trials.
Why it matters
Mistral AI presents the model as a way to open up research into architectures beyond the transformer, following its Mixtral releases. Its open licence means others can study, adapt and redistribute it. The company also expects it to work well as a local coding helper, and points to the efficiency of the design as useful for code work where long inputs are common.
Questions people ask about Codestral Mamba
What makes Codestral Mamba different from a transformer model?
It uses the Mamba2 state-space design rather than attention. Mistral AI says this gives linear-time inference, so replies stay fast no matter how long the input is, and in principle supports sequences of unlimited length. The company tested in-context retrieval at up to 256k tokens.
How large is it and what is it trained to do?
The instruction-tuned model has 7,285,403,648 parameters. Training concentrated on code and reasoning, and Mistral AI positions it as a local coding assistant. The company claims it performs on a par with leading transformer code models of similar size.
How can I run it?
Weights are on Hugging Face under Apache 2.0. Mistral AI recommends its mistral-inference SDK, which builds on the reference Mamba code. TensorRT-LLM works too, and llama.cpp support was described as coming. A hosted copy is offered on la Plateforme as codestral-mamba-2407.
How does it relate to Codestral 22B?
Codestral 22B is the larger sibling in the same family. Mistral AI's evaluation table shows it scoring above Codestral Mamba on most listed tests. Licensing differs: Codestral Mamba is Apache 2.0, while Codestral 22B needs a commercial licence for self-deployment or a community licence for testing.
Good reading and sources
There is no English Wikipedia article on Codestral Mamba yet, so the facts on this page come from the sources listed above: mistral.ai/news/codestral-mamba, huggingface.co/mistralai/Mamba-Codestral-7B-v0.1. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.