Grok-1
xAI's 314 billion parameter open-weights mixture-of-experts base model
Grok-1 is a large language model built by xAI, trained from scratch and released with its weights and architecture in March 2024. It is a 314 billion parameter Mixture-of-Experts base model whose pre-training finished in October 2023, and it was published under the Apache 2.0 licence as a raw checkpoint not tuned for any particular task.
- 314 billion parameter Mixture-of-Experts model
- 25% of the weights active on a given token
- Pre-training concluded in October 2023
- Weights and architecture released under Apache 2.0
- Raw base model checkpoint, not fine-tuned for dialogue
What it is
Grok-1 is a large language model created by xAI. It is a Mixture-of-Experts system with 314 billion parameters, meaning only part of the network is used for any single token: xAI says 25% of the weights are active on a given token. The company released the weights and the network architecture openly. What was published is the raw checkpoint from the pre-training stage, which ended in October 2023. Because it comes straight from pre-training, the model has not been fine-tuned for a specific job such as conversation. xAI describes it as a base model trained on a large quantity of text and not adapted to any particular task.
How it came about
xAI trained Grok-1 from scratch, finishing the pre-training phase in October 2023. The company then published the base model checkpoint together with the architecture, releasing the weights under the Apache 2.0 licence and pointing users to a GitHub repository for setup instructions. The weights were also made available through a Hugging Face repository under the name xai-org/grok-1, with an int8 checkpoint offered for download.
Inside the model
Grok-1 is a Mixture-of-Experts model, a design where the network is split into many expert sub-networks and only a fraction is consulted for each token. xAI states that 25% of the weights are active on a given token, out of 314 billion parameters in total. Training used a custom stack built on top of JAX and Rust, and the model was trained from scratch on a large amount of text data. The released checkpoint is the raw pre-training output, so no dialogue tuning or other fine-tuning has been applied to it.
What it can do
Grok-1 is offered as a general base model rather than a tuned assistant. Because the released checkpoint is not fine-tuned for any application, including dialogue, it is intended as a starting point rather than a ready-made chatbot. Running the example code requires a multi-GPU machine given the model's size of 314 billion parameters.
How to use it
The weights and architecture are published under the Apache 2.0 licence. Users can clone the GitHub repository and download the int8 checkpoint from the xai-org/grok-1 repository on Hugging Face. Setup instructions and requirements are provided in the repository, and a multi-GPU machine is needed to run the example code.
Questions people ask about Grok-1
What is Grok-1?
Grok-1 is a large language model from xAI with 314 billion parameters, built as a Mixture-of-Experts system. The company released its weights and architecture openly, and describes it as a base model trained from scratch on a large amount of text data.
Is Grok-1 a chatbot?
No. The released checkpoint is the raw output of the pre-training phase, which ended in October 2023. xAI says it has not been fine-tuned for any specific application, including dialogue, so it is a base model rather than a conversational assistant.
How can I use Grok-1?
The weights are available under the Apache 2.0 licence through the xai-org/grok-1 repository on Hugging Face, with code on GitHub. You download the int8 checkpoint and follow the setup instructions. A multi-GPU machine is required because the model has 314 billion parameters.
What does Mixture-of-Experts mean here?
It is a design where the network is divided into expert sub-networks and only some are used for each token. xAI states that 25% of Grok-1's weights are active on a given token, out of 314 billion parameters in total.
Good reading and sources
There is no English Wikipedia article on Grok-1 yet, so the facts on this page come from the sources listed above: x.ai/news/grok-os, huggingface.co/xai-org/grok-1. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.