1.58-bit large language model
A model whose weights hold only three values: -1, 0 and +1.
A 1.58-bit large language model, also called a ternary LLM, keeps every weight to one of three values: -1, 0 or +1. Microsoft researchers introduced the BitNet line in 2023 and 2024, and released an open-weights 2-billion-parameter version in 2025, showing that such models can match full-precision ones while using far less memory.
- Weights take only three values: -1, 0 and +1
- Three states carry about 1.58 bits, hence the name
- BitLinear replaces the standard linear layer in the transformer
- BitNet b1.58 2B4T: 2B parameters, 4T tokens, open weights
- Multiplications become additions, cutting memory and cost
What it is
A 1.58-bit large language model is a design that stores each weight as one of just three values: -1, 0 or +1. Because three states carry about 1.58 bits of information, the approach is also called ternary. Compared with the 16-bit floating-point numbers used by ordinary models, this cuts memory use sharply and lets multiplications be swapped for cheaper additions. Researchers at Microsoft set out the idea in the BitNet work from 2023 and 2024, and in 2025 shipped an open-weights model of 2 billion parameters trained on 4 trillion tokens.
How it came about
The line began with BitNet, described by Hongyu Wang and colleagues at Microsoft in a 2023 paper on scaling 1-bit transformers. In February 2024 a team led by Shuming Ma announced BitNet b1.58, reporting results on a par with the 16-bit Llama 2 and calling it the start of the 1-bit LLM era. A 2025 technical report from Microsoft covered BitNet b1.58 2B4T, a 2-billion-parameter model trained on 4 trillion tokens, released with open weights and open inference code.
Inside the model
Rather than training at full precision and shrinking the numbers afterwards, BitNet is built for 1.58 bits from the start. Its key change is the BitLinear transform, which takes the place of the ordinary linear layer inside the transformer. Weights are limited to -1, 0 and +1, so the heavy multiplications that dominate normal networks become additions. The name reflects the information content of three states, roughly log2(3), about 1.58 bits. Papers sometimes call these 1-bit models, though that phrase can also mean true binary weights of -1 and +1.
What it can do
For models up to a few billion parameters, studies report that 1.58-bit systems perform about as well as their full-precision versions across a range of tasks. BitNet b1.58 was described as matching 16-bit Llama 2, and the 2-billion-parameter 2B4T model was reported as competitive with full-precision models of its size. Critics note that the gains depend on training budget: scaling-law work argues low-bit weights only come out ahead for undertrained models, and that more training tokens expose their weaknesses.
How to use it
BitNet b1.58 2B4T was published with open weights and open inference code. The wider 1.58-bit approach is described in public research papers, including the BitNet reports and a 2024 method from HuggingFace for gradually pushing an existing model down to 1.58 bits during fine-tuning.
Reception and debate
The BitNet papers drew attention for showing that very low-precision weights could hold up against standard 16-bit models, and coverage in IEEE Spectrum and Tom's Hardware highlighted the promise of running capable AI on cheaper, lower-power hardware. Not all researchers agree: work on scaling laws for precision argues the advantage is limited to undertrained models, and that heavy training erodes it.
Why it matters
The 1.58-bit line is presented as a route to capable models on ordinary, energy-efficient hardware rather than specialised accelerators. It also shifted attention from post-training quantisation towards training natively at low precision, and prompted follow-up work on converting existing models, such as HuggingFace's gradual quantisation during fine-tuning.
Questions people ask about 1.58-bit large language model
Why is it called 1.58-bit?
A weight with three possible states holds log2(3) bits of information, which is about 1.58. The name describes how much information each weight stores, not a hardware word size. Papers also use the phrase 1-bit LLM, though that can mean true binary weights of -1 and +1.
How is it different from quantising a normal model?
BitNet is trained natively at 1.58 bits instead of being shrunk from a full-precision model afterwards. The BitLinear transform replaces the ordinary linear layer. That said, HuggingFace showed in 2024 how to ramp quantisation up gradually while fine-tuning an existing model down to 1.58 bits.
Does it really match full-precision models?
For models of up to a few billion parameters, studies report performance comparable to full-precision counterparts. BitNet b1.58 was reported as matching 16-bit Llama 2, and the 2-billion-parameter 2B4T model as competitive at its size. Critics argue the advantage fades as training tokens increase.
What is the benefit in practice?
Restricting weights to three values cuts the memory footprint and lets expensive multiplications be replaced with cheaper additions. That opens the door to running capable models on less specialised, lower-power hardware rather than costly accelerators.
Good reading and sources
- The Era of 1-bit LLMs: All Large Language Models are in 1.58 BitsarXiv, Article
- Microsoft researchers build 1-bit AI LLM with 2B parameters — model small enough to run on some CPUsTom's Hardware, Article
- 1-bit LLMs Could Solve AI’s Energy DemandsIEEE Spectrum, Article
- Fine-tuning LLMs to 1.58bit: extreme quantization made easyHugging Face, Article
The facts on this page come from the Wikipedia article "1.58-bit large language model", written by its contributors and used under the CC BY-SA 4.0 licence, and from its Wikidata record (CC0). EveryLLM rewrote the text in its own words, with AI assistance, and compares every page with the article to make sure no passage is copied. It may be reused under the same CC BY-SA 4.0 licence with a link back.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.