News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
DBRX

DBRX

A 132-billion-parameter mixture-of-experts language model from Databricks.

DBRX is a large language model built by Mosaic under its parent company Databricks, released on 27 March 2024. It is a mixture-of-experts transformer with 132 billion parameters, of which 36 billion are active per token, and it was offered as both a base model and an instruction-tuned variant.

Share on WhatsApp

  • 132 billion parameters in total, 36 billion active per token
  • Mixture-of-experts transformer with 16 experts, 4 used per token
  • Released 27 March 2024 under the Databricks Open Model License
  • Trained on 3,072 Nvidia H100s over about 2.5 months
  • Reported training cost of US$10 million

What it is

DBRX is a large language model released by Databricks on 27 March 2024. It was built by Mosaic, which sits under the Databricks parent company. The model uses a mixture-of-experts transformer design and holds 132 billion parameters in total, with 36 billion of them active for any single token. That means only a slice of the network works on each word it produces. Two versions were published: a base foundation model and an instruction-tuned variant, which is the form adjusted to follow user requests. It is a general-purpose text model.

How it came about

Mosaic, working under parent company Databricks, developed DBRX and put it out on 27 March 2024. It was published under the Databricks Open Model License, with code and weights hosted in a public repository. Training ran for roughly two and a half months. The release was presented as an open model that could stand alongside well-known rivals.

Inside the model

DBRX is a mixture-of-experts transformer. In this design the network is split into 16 separate expert blocks, and the model picks 4 of them to handle each token, so 36 billion parameters are active out of 132 billion in total. Training used 3,072 Nvidia H100 chips linked by InfiniBand connections offering 3.2 terabytes per second of bandwidth. The whole run took about 2.5 months and was reported to cost US$10 million. The model was released both as a base foundation model and as an instruction-tuned version.

What it can do

On release, DBRX was reported to beat several well-known open models across a range of tests. These covered language understanding, programming and mathematics. The models it was compared against included Meta's Llama 2, Mistral AI's Mixtral and xAI's Grok-1. No individual benchmark figures are given here.

How to use it

DBRX was published under the Databricks Open Model License, and its code and weights are available in a public GitHub repository. It comes in two forms: a base foundation model and an instruction-tuned variant. The model can also be reached through Databricks' own platform.

Questions people ask about DBRX

What is DBRX?

DBRX is a large language model released by Databricks on 27 March 2024. It was built by Mosaic, part of Databricks. It is a mixture-of-experts transformer with 132 billion parameters, and it was published as both a base foundation model and an instruction-tuned variant.

What does mixture-of-experts mean here?

The network is divided into 16 expert blocks. For each token it produces, the model uses only 4 of them. That keeps 36 billion parameters active out of 132 billion in total, so less computation is needed per token than the full size suggests.

How was DBRX trained?

Training ran for about 2.5 months on 3,072 Nvidia H100 chips joined by InfiniBand links with 3.2 terabytes per second of bandwidth. The reported cost of the run was US$10 million.

How can I use DBRX?

It was released under the Databricks Open Model License, with code and weights in a public GitHub repository. Two versions exist: a base foundation model and an instruction-tuned one. It is also available through Databricks' platform.

How does it compare to other models?

At the time of release it was reported to outperform Llama 2, Mixtral and Grok-1 on several benchmarks covering language understanding, programming and mathematics.

Good reading and sources

Where this page comes from

The facts on this page come from the Wikipedia article "DBRX", written by its contributors and used under the CC BY-SA 4.0 licence, and from its Wikidata record (CC0). EveryLLM rewrote the text in its own words, with AI assistance, and compares every page with the article to make sure no passage is copied. It may be reused under the same CC BY-SA 4.0 licence with a link back.

It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

The weekly digest

Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.