StableLM 2
A small open-weight language model by Stability AI
Stability AI released StableLM 2 1.6B in February 2024 as the first model in a new series. It is a decoder-only transformer with 1.6 billion parameters trained on two trillion tokens. The company claims it was the leading open model under two billion parameters at launch.
- Contains 1.6 billion parameters
- Trained on 2 trillion tokens
- Supports multilingual and code tasks
- Uses Stability AI Community License
- Optimized for edge device deployment
What it is
StableLM 2 1.6B is a foundational language model developed by Stability AI. Released in early 2024, it features 1.6 billion parameters and serves as the initial entry in a new generation of the company’s models. The system is designed for text generation and acts as a base for further fine-tuning. It supports multiple languages and coding tasks. Stability AI states that this model set a new performance standard for open models with fewer than two billion parameters at the time of its release.
How it came about
The technical report for StableLM 2 1.6B was published on arXiv in February 2024. This release marked the start of a new series from Stability AI. The team made both base and instruction-tuned versions available for download via Hugging Face. The announcement included detailed documentation on the training data and procedures. The developers aimed to provide a high-performance option for users with limited computational resources.
Inside the model
The model uses a decoder-only transformer architecture similar to LLaMA but with specific changes. It has 24 layers, a hidden size of 2048, and 32 attention heads. Positional information is handled by Rotary Position Embeddings applied to a quarter of the head dimensions. Normalization uses LayerNorm with learned biases rather than RMSNorm. Most bias terms were removed from feed-forward and attention layers, except for query, key, and value projections. Training used the Arcade100k tokenizer on two trillion tokens of multilingual and code data.
What it can do
Stability AI claims the model was state-of-the-art among open models under two billion parameters when launched. Evaluations covered zero-shot and few-shot benchmarks, multilingual tasks, and multi-turn dialogue. The small size allows for efficient throughput on edge devices. The company also released quantized checkpoints to further reduce resource needs. Users are advised to fine-tune the base model for specific downstream applications to ensure safe and reliable performance.
How to use it
The weights for both base and instruction-tuned versions are available on Hugging Face. They are distributed under the Stability AI Community License. Commercial use requires adherence to separate licensing terms provided by the company. The model can be run using standard transformer libraries. Support for Flash Attention 2 is included to improve inference speed. Quantized versions are also open-sourced for easier deployment on hardware with limited memory.
Safety and testing
As a base model, it may produce unsafe or unreliable outputs. The pre-training data underwent filtering but might still contain offensive content. Developers must evaluate and fine-tune the model before deploying it in production. The company warns against using the model for applications that could cause harm. Caution is recommended when integrating the raw base model into user-facing systems without additional safety alignment.
Questions people ask about StableLM 2
What is the parameter count of StableLM 2?
The model has approximately 1.6 billion parameters. Specifically, the architecture contains 1,644,417,024 parameters across 24 layers.
Can I use StableLM 2 for commercial projects?
Yes, but you must refer to the specific commercial license terms provided by Stability AI. The default weights are released under the Stability AI Community License.
What hardware was used to train this model?
Training took place on the Stability AI cluster using 512 NVIDIA A100 GPUs with 40GB of memory each. These were hosted on AWS P4d instances.
Is StableLM 2 suitable for running on local devices?
Yes, its small size makes it appealing for edge devices. The developers provided throughput measurements and quantized checkpoints to facilitate local deployment.
Good reading and sources
There is no English Wikipedia article on StableLM 2 yet, so the facts on this page come from the sources listed above: huggingface.co/stabilityai/stablelm-2-1_6b, arxiv.org/abs/2402.17834. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.