News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
StableLM 2

StableLM 2

A small open-weight language model by Stability AI

Stability AI released StableLM 2 1.6B in February 2024 as the first model in a new series. It is a decoder-only transformer with 1.6 billion parameters trained on two trillion tokens. The company claims it was the leading open model under two billion parameters at launch.

Share on WhatsApp

  • Contains 1.6 billion parameters
  • Trained on 2 trillion tokens
  • Supports multilingual and code tasks
  • Uses Stability AI Community License
  • Optimized for edge device deployment

What it is

StableLM 2 1.6B is a foundational language model developed by Stability AI. Released in early 2024, it features 1.6 billion parameters and serves as the initial entry in a new generation of the company’s models. The system is designed for text generation and acts as a base for further fine-tuning. It supports multiple languages and coding tasks. Stability AI states that this model set a new performance standard for open models with fewer than two billion parameters at the time of its release.

How it came about

The technical report for StableLM 2 1.6B was published on arXiv in February 2024. This release marked the start of a new series from Stability AI. The team made both base and instruction-tuned versions available for download via Hugging Face. The announcement included detailed documentation on the training data and procedures. The developers aimed to provide a high-performance option for users with limited computational resources.

Inside the model

The model uses a decoder-only transformer architecture similar to LLaMA but with specific changes. It has 24 layers, a hidden size of 2048, and 32 attention heads. Positional information is handled by Rotary Position Embeddings applied to a quarter of the head dimensions. Normalization uses LayerNorm with learned biases rather than RMSNorm. Most bias terms were removed from feed-forward and attention layers, except for query, key, and value projections. Training used the Arcade100k tokenizer on two trillion tokens of multilingual and code data.

What it can do

Stability AI claims the model was state-of-the-art among open models under two billion parameters when launched. Evaluations covered zero-shot and few-shot benchmarks, multilingual tasks, and multi-turn dialogue. The small size allows for efficient throughput on edge devices. The company also released quantized checkpoints to further reduce resource needs. Users are advised to fine-tune the base model for specific downstream applications to ensure safe and reliable performance.

How to use it

The weights for both base and instruction-tuned versions are available on Hugging Face. They are distributed under the Stability AI Community License. Commercial use requires adherence to separate licensing terms provided by the company. The model can be run using standard transformer libraries. Support for Flash Attention 2 is included to improve inference speed. Quantized versions are also open-sourced for easier deployment on hardware with limited memory.

Safety and testing

As a base model, it may produce unsafe or unreliable outputs. The pre-training data underwent filtering but might still contain offensive content. Developers must evaluate and fine-tune the model before deploying it in production. The company warns against using the model for applications that could cause harm. Caution is recommended when integrating the raw base model into user-facing systems without additional safety alignment.

Questions people ask about StableLM 2

What is the parameter count of StableLM 2?

The model has approximately 1.6 billion parameters. Specifically, the architecture contains 1,644,417,024 parameters across 24 layers.

Can I use StableLM 2 for commercial projects?

Yes, but you must refer to the specific commercial license terms provided by Stability AI. The default weights are released under the Stability AI Community License.

What hardware was used to train this model?

Training took place on the Stability AI cluster using 512 NVIDIA A100 GPUs with 40GB of memory each. These were hosted on AWS P4d instances.

Is StableLM 2 suitable for running on local devices?

Yes, its small size makes it appealing for edge devices. The developers provided throughput measurements and quantized checkpoints to facilitate local deployment.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on StableLM 2 yet, so the facts on this page come from the sources listed above: huggingface.co/stabilityai/stablelm-2-1_6b, arxiv.org/abs/2402.17834. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.