News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
1.58-bit large language model

1.58-bit large language model

A model whose weights hold only three values: -1, 0 and +1.

A 1.58-bit large language model, also called a ternary LLM, keeps every weight to one of three values: -1, 0 or +1. Microsoft researchers introduced the BitNet line in 2023 and 2024, and released an open-weights 2-billion-parameter version in 2025, showing that such models can match full-precision ones while using far less memory.

Share on WhatsApp

  • Weights take only three values: -1, 0 and +1
  • Three states carry about 1.58 bits, hence the name
  • BitLinear replaces the standard linear layer in the transformer
  • BitNet b1.58 2B4T: 2B parameters, 4T tokens, open weights
  • Multiplications become additions, cutting memory and cost

What it is

A 1.58-bit large language model is a design that stores each weight as one of just three values: -1, 0 or +1. Because three states carry about 1.58 bits of information, the approach is also called ternary. Compared with the 16-bit floating-point numbers used by ordinary models, this cuts memory use sharply and lets multiplications be swapped for cheaper additions. Researchers at Microsoft set out the idea in the BitNet work from 2023 and 2024, and in 2025 shipped an open-weights model of 2 billion parameters trained on 4 trillion tokens.

How it came about

The line began with BitNet, described by Hongyu Wang and colleagues at Microsoft in a 2023 paper on scaling 1-bit transformers. In February 2024 a team led by Shuming Ma announced BitNet b1.58, reporting results on a par with the 16-bit Llama 2 and calling it the start of the 1-bit LLM era. A 2025 technical report from Microsoft covered BitNet b1.58 2B4T, a 2-billion-parameter model trained on 4 trillion tokens, released with open weights and open inference code.

Inside the model

Rather than training at full precision and shrinking the numbers afterwards, BitNet is built for 1.58 bits from the start. Its key change is the BitLinear transform, which takes the place of the ordinary linear layer inside the transformer. Weights are limited to -1, 0 and +1, so the heavy multiplications that dominate normal networks become additions. The name reflects the information content of three states, roughly log2(3), about 1.58 bits. Papers sometimes call these 1-bit models, though that phrase can also mean true binary weights of -1 and +1.

What it can do

For models up to a few billion parameters, studies report that 1.58-bit systems perform about as well as their full-precision versions across a range of tasks. BitNet b1.58 was described as matching 16-bit Llama 2, and the 2-billion-parameter 2B4T model was reported as competitive with full-precision models of its size. Critics note that the gains depend on training budget: scaling-law work argues low-bit weights only come out ahead for undertrained models, and that more training tokens expose their weaknesses.

How to use it

BitNet b1.58 2B4T was published with open weights and open inference code. The wider 1.58-bit approach is described in public research papers, including the BitNet reports and a 2024 method from HuggingFace for gradually pushing an existing model down to 1.58 bits during fine-tuning.

Reception and debate

The BitNet papers drew attention for showing that very low-precision weights could hold up against standard 16-bit models, and coverage in IEEE Spectrum and Tom's Hardware highlighted the promise of running capable AI on cheaper, lower-power hardware. Not all researchers agree: work on scaling laws for precision argues the advantage is limited to undertrained models, and that heavy training erodes it.

Why it matters

The 1.58-bit line is presented as a route to capable models on ordinary, energy-efficient hardware rather than specialised accelerators. It also shifted attention from post-training quantisation towards training natively at low precision, and prompted follow-up work on converting existing models, such as HuggingFace's gradual quantisation during fine-tuning.

Questions people ask about 1.58-bit large language model

Why is it called 1.58-bit?

A weight with three possible states holds log2(3) bits of information, which is about 1.58. The name describes how much information each weight stores, not a hardware word size. Papers also use the phrase 1-bit LLM, though that can mean true binary weights of -1 and +1.

How is it different from quantising a normal model?

BitNet is trained natively at 1.58 bits instead of being shrunk from a full-precision model afterwards. The BitLinear transform replaces the ordinary linear layer. That said, HuggingFace showed in 2024 how to ramp quantisation up gradually while fine-tuning an existing model down to 1.58 bits.

Does it really match full-precision models?

For models of up to a few billion parameters, studies report performance comparable to full-precision counterparts. BitNet b1.58 was reported as matching 16-bit Llama 2, and the 2-billion-parameter 2B4T model as competitive at its size. Critics argue the advantage fades as training tokens increase.

What is the benefit in practice?

Restricting weights to three values cuts the memory footprint and lets expensive multiplications be replaced with cheaper additions. That opens the door to running capable models on less specialised, lower-power hardware rather than costly accelerators.

Good reading and sources

Where this page comes from

The facts on this page come from the Wikipedia article "1.58-bit large language model", written by its contributors and used under the CC BY-SA 4.0 licence, and from its Wikidata record (CC0). EveryLLM rewrote the text in its own words, with AI assistance, and compares every page with the article to make sure no passage is copied. It may be reused under the same CC BY-SA 4.0 licence with a link back.

It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

The weekly digest

Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.