Granite 4.0
IBM's hybrid Mamba/transformer language models for enterprise work
Granite 4.0 is a family of open language models from IBM's Granite Team, released on 2 October 2025. It mixes Mamba-2 layers with a small amount of transformer attention to cut memory use, and ships in several sizes for enterprise agents, edge devices and local use.
- Released 2 October 2025 under Apache 2.0 by IBM's Granite Team
- H-Small: 32B total parameters, 9B active per token
- Hybrid design mixes Mamba-2 layers with a minority of attention layers
- IBM claims over 70% less RAM for long inputs and concurrent batches
- First open models IBM says are ISO 42001 certified, with signed checkpoints
What it is
Granite 4.0 is IBM's newest line of open language models, aimed at business use. The collection spans several sizes: H-Small carries 32 billion parameters with 9 billion active per token, H-Tiny has 7 billion total and 1 billion active, and two 3 billion dense variants round out the set. IBM positions the larger one for multi-tool agents and support automation, and the smaller ones for low-latency, on-device work. The models answer general instructions, handle retrieval-augmented generation, multilingual chat and code completion, and can call external functions and APIs. Weights are published under Apache 2.0.
How it came about
IBM's Granite Team built the family and put it out on 2 October 2025, with Base and Instruct versions of the Micro, Tiny and Small models. Larger and smaller sizes, plus variants with explicit reasoning, were promised before the end of 2025. Firms including EY and Lockheed Martin got early access to try the models at scale, and their comments, along with community feedback, are meant to shape later updates. On 7 October 2025 a default system prompt was added to the chat template of H-Small to steer replies toward professional, accurate and safe output.
Inside the model
The hybrid builds pair a minority of ordinary transformer attention layers with a majority of Mamba-2 layers, a sequence model that handles language differently and more cheaply than attention alone. H-Small is a decoder-only mixture-of-experts setup using grouped-query attention, shared experts, SwiGLU activation, RMSNorm and tied input/output embeddings. Supervised fine-tuning drew on permissively licensed public datasets, internal synthetic data aimed at specific skills and a small set of human-curated pairs, followed by reinforcement-learning alignment and model merging. Training ran on an NVIDIA GB200 NVL72 cluster at CoreWeave, linked by NVLink within a rack and a non-blocking Fat-Tree NDR 400 Gb/s InfiniBand fabric between racks.
What it can do
IBM reports sizeable gains over Granite 3.3, saying even the smallest 4.0 models beat the 8 billion parameter Granite 3.3 while being under half its size. The clearest claimed edge is inference efficiency: the hybrid designs need far less memory, and IBM says RAM use for long inputs and concurrent batches drops by more than 70% against standard transformers. Throughput is said to hold up as context and batch size grow. Instruct versions improve instruction following and tool calling. Fine-tuning is mostly English, so multilingual results may lag, and a few-shot examples can help.
How to use it
Weights are downloadable under Apache 2.0 from Hugging Face, with code on GitHub and guidance in IBM's Granite docs. The models also run on IBM watsonx.ai and through partners such as Dell Technologies, Docker Hub, Kaggle, LM Studio, NVIDIA NIM, Ollama, OPAQUE and Replicate, with Amazon SageMaker JumpStart and Microsoft Azure AI Foundry announced as coming. Checkpoints on Hugging Face are cryptographically signed so users can verify where a model came from.
Safety and testing
IBM says an outside audit lasting several months examined its AI development process, after which Granite became the only open language model family with ISO 42001 certification, a standard covering accountability, explainability, data privacy and reliability in AI management. A bug bounty programme with HackerOne covers Granite. The company still warns that aligned models can give inaccurate, biased or unsafe answers and asks users to run their own safety testing and tuning for their tasks.
Reception and debate
Enterprise partners EY and Lockheed Martin were among those given early access to test the models on key use cases at scale. IBM says feedback from these partners and from the open-source community will feed into future updates. The company presents the release as the start of a new phase for its enterprise language model family, built around small, efficient models.
Why it matters
IBM frames the line as a way to lower the cost of running capable models, letting them serve as cheap building blocks beside larger reasoning systems in agentic workflows. The company worked with Qualcomm Technologies and Nexa AI on the models, and says the hybrid designs run on AMD Instinct MI-300X GPUs as well, further trimming their memory footprint. The family is described as the first open models to carry ISO 42001 certification.
Questions people ask about Granite 4.0
What sizes does Granite 4.0 come in?
The launch set has four: H-Small with 32 billion parameters (9 billion active), H-Tiny with 7 billion (1 billion active), and two 3 billion dense models, H-Micro and Micro. Base and Instruct versions of Micro, Tiny and Small shipped at release, with more sizes promised.
Why does the hybrid design matter?
Most layers use Mamba-2 rather than standard attention, with only a few transformer attention layers mixed in. IBM says this keeps memory needs low, especially for long inputs and many simultaneous sessions, so the models can run on cheaper GPUs and stay fast under heavy load.
How do I get and use the models?
Weights are on Hugging Face under Apache 2.0, with a GitHub repository and IBM Granite docs for guidance. They also run on IBM watsonx.ai and partner platforms including Docker Hub, Kaggle, LM Studio, NVIDIA NIM, Ollama and Replicate.
What languages does it handle?
The listed set covers English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch and Chinese. Fine-tuning is mostly English, so other languages may perform less well, and IBM suggests adding a few examples to the prompt to improve output.
What about safety and trust?
IBM says an external audit led to ISO 42001 certification for its AI management system, and it runs a HackerOne bug bounty for Granite. Checkpoints are cryptographically signed. The company still notes models can produce inaccurate, biased or unsafe replies and asks users to test and tune for their own tasks.
Good reading and sources
There is no English Wikipedia article on Granite 4.0 yet, so the facts on this page come from the sources listed above: huggingface.co/ibm-granite/granite-4.0-h-small, ibm.com/new/announcements/ibm-granite-4-0-hyper-efficient-high-perform. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.