News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
Grok-1

Grok-1

xAI's 314 billion parameter open-weights mixture-of-experts base model

Grok-1 is a large language model built by xAI, trained from scratch and released with its weights and architecture in March 2024. It is a 314 billion parameter Mixture-of-Experts base model whose pre-training finished in October 2023, and it was published under the Apache 2.0 licence as a raw checkpoint not tuned for any particular task.

Share on WhatsApp

  • 314 billion parameter Mixture-of-Experts model
  • 25% of the weights active on a given token
  • Pre-training concluded in October 2023
  • Weights and architecture released under Apache 2.0
  • Raw base model checkpoint, not fine-tuned for dialogue

What it is

Grok-1 is a large language model created by xAI. It is a Mixture-of-Experts system with 314 billion parameters, meaning only part of the network is used for any single token: xAI says 25% of the weights are active on a given token. The company released the weights and the network architecture openly. What was published is the raw checkpoint from the pre-training stage, which ended in October 2023. Because it comes straight from pre-training, the model has not been fine-tuned for a specific job such as conversation. xAI describes it as a base model trained on a large quantity of text and not adapted to any particular task.

How it came about

xAI trained Grok-1 from scratch, finishing the pre-training phase in October 2023. The company then published the base model checkpoint together with the architecture, releasing the weights under the Apache 2.0 licence and pointing users to a GitHub repository for setup instructions. The weights were also made available through a Hugging Face repository under the name xai-org/grok-1, with an int8 checkpoint offered for download.

Inside the model

Grok-1 is a Mixture-of-Experts model, a design where the network is split into many expert sub-networks and only a fraction is consulted for each token. xAI states that 25% of the weights are active on a given token, out of 314 billion parameters in total. Training used a custom stack built on top of JAX and Rust, and the model was trained from scratch on a large amount of text data. The released checkpoint is the raw pre-training output, so no dialogue tuning or other fine-tuning has been applied to it.

What it can do

Grok-1 is offered as a general base model rather than a tuned assistant. Because the released checkpoint is not fine-tuned for any application, including dialogue, it is intended as a starting point rather than a ready-made chatbot. Running the example code requires a multi-GPU machine given the model's size of 314 billion parameters.

How to use it

The weights and architecture are published under the Apache 2.0 licence. Users can clone the GitHub repository and download the int8 checkpoint from the xai-org/grok-1 repository on Hugging Face. Setup instructions and requirements are provided in the repository, and a multi-GPU machine is needed to run the example code.

Questions people ask about Grok-1

What is Grok-1?

Grok-1 is a large language model from xAI with 314 billion parameters, built as a Mixture-of-Experts system. The company released its weights and architecture openly, and describes it as a base model trained from scratch on a large amount of text data.

Is Grok-1 a chatbot?

No. The released checkpoint is the raw output of the pre-training phase, which ended in October 2023. xAI says it has not been fine-tuned for any specific application, including dialogue, so it is a base model rather than a conversational assistant.

How can I use Grok-1?

The weights are available under the Apache 2.0 licence through the xai-org/grok-1 repository on Hugging Face, with code on GitHub. You download the int8 checkpoint and follow the setup instructions. A multi-GPU machine is required because the model has 314 billion parameters.

What does Mixture-of-Experts mean here?

It is a design where the network is divided into expert sub-networks and only some are used for each token. xAI states that 25% of Grok-1's weights are active on a given token, out of 314 billion parameters in total.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Grok-1 yet, so the facts on this page come from the sources listed above: x.ai/news/grok-os, huggingface.co/xai-org/grok-1. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.