News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
INTELLECT-1

INTELLECT-1

A 10 billion parameter model trained by a global community.

INTELLECT-1 is a base language model with 10 billion parameters. Prime Intellect and thirty community contributors trained it on one trillion tokens. Released in November 2024, it proves large models can be built using distributed, non-corporate compute resources across three continents.

Share on WhatsApp

  • Trained on 1 trillion tokens of English text and code.
  • Used compute from 30 independent global contributors.
  • Reduced communication overhead by 400x using int8 kernels.
  • Maintained 83-96% compute utilisation across unstable nodes.
  • Released under an Apache 2.0 open licence.

What it is

INTELLECT-1 is a base language model containing ten billion parameters. It processes English text and code. The model was released on 29 November 2024 under an Apache 2.0 licence. Its primary significance lies in its training method. Rather than relying on a single corporate data centre, the project used compute power donated by thirty independent providers. These contributors operated across three continents. The model serves as a foundation for further instruction tuning, with a separate chat version available for conversational tasks.

How it came about

The project was led by Prime Intellect. A technical report authored by Sami Jaghouar and colleagues details the development. Training occurred on up to fourteen concurrent nodes. Thirty independent compute providers joined and left the process dynamically. This collaborative effort aimed to show that frontier foundation models do not require centralised corporate infrastructure. The team published their findings in December 2024 to document the feasibility of this decentralised approach.

Inside the model

The model uses a decoder-only transformer architecture with thirty-two attention heads. Training relied on the PRIME framework, which handles fault tolerance for unreliable global workers. A key component is the ElasticDeviceMesh. This system manages communication groups both within local nodes and across the internet. The team employed DiLoCo algorithms with one hundred inner steps. They also used custom int8 all-reduce kernels. This technique reduced communication bandwidth by a factor of four hundred compared to standard methods. The training data comprised one trillion tokens, including FineWeb, Stack V1, and Open Web Math.

What it can do

As a base model, INTELLECT-1 scores 37.5 on MMLU and 52.13 on ARC-C. Its GSM8K score is 8.1. The instruction-tuned variant performs better, reaching 49.89 on MMLU and 38.58 on GSM8K. These results are comparable to other models in the seven to thirteen billion parameter range. The model requires a beginning-of-sequence token at the start of each input sample. Omitting this token may reduce performance.

How to use it

The weights are openly available under the Apache 2.0 licence. Users can download the base model or the instruction-tuned version from Hugging Face. The code for generation is compatible with standard transformer libraries. The training framework itself is also open source. This allows other researchers to replicate the distributed training setup using their own hardware contributions.

Why it matters

INTELLECT-1 demonstrates that high-performance model training is possible without centralised resources. It validates the use of dynamic, globally distributed GPU networks. The project highlights the potential for community-driven AI development. By achieving high compute utilisation despite unreliable nodes, it offers a blueprint for future decentralised training efforts.

Questions people ask about INTELLECT-1

Who trained INTELLECT-1?

Prime Intellect led the project, but thirty independent community contributors provided the compute power. These providers were located across three continents and joined the training process dynamically.

What is the PRIME framework?

PRIME is a scalable distributed training framework designed for fault tolerance. It uses ElasticDeviceMesh to manage communication between unreliable global workers, allowing dynamic scaling during training.

Is INTELLECT-1 a chat model?

No, INTELLECT-1 is a base model. For chat applications, users should use INTELLECT-1-Instruct, which is a separate version fine-tuned for conversational tasks.

How large is the model?

The model has ten billion parameters. It was trained on one trillion tokens of data, including sources like FineWeb, Stack V1, and Open Web Math.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on INTELLECT-1 yet, so the facts on this page come from the sources listed above: huggingface.co/PrimeIntellect/INTELLECT-1, arxiv.org/abs/2412.01152. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.