News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
Falcon 3

Falcon 3

Open foundation models from 1B to 10B parameters by TII

Falcon 3 is a family of open large language models released in December 2024 by the Technology Innovation Institute. Ranging from 1 billion to 10 billion parameters, these models support English, French, Spanish and Portuguese with a 32,000 token context window.

Share on WhatsApp

  • Released in December 2024 by Technology Innovation Institute
  • Supports English, French, Spanish and Portuguese
  • Context length extends up to 32,000 tokens
  • Trained on 2 teratokens using 1,024 H100 GPUs
  • Uses Grouped Query Attention for faster inference

What it is

Falcon 3 is a collection of pretrained and instruction-tuned large language models developed by the Technology Innovation Institute. The family includes sizes from 1 billion to 10 billion parameters. Released in December 2024, these models aim to perform well in reasoning, language understanding and code generation. They support four languages: English, French, Spanish and Portuguese. The largest model handles context windows up to 32,000 tokens, allowing it to process long documents or extended conversations without losing track of earlier information.

How it came about

The Technology Innovation Institute built the Falcon 3 family as a set of open foundation models. The release occurred in December 2024. This series follows previous Falcon efforts, scaling up depth and capability. The team trained the base models on extensive datasets before creating specific instruct versions for chat and task completion. The 10-billion parameter instruct model represents the flagship release in this particular batch, designed to balance size with high performance on standard benchmarks.

Inside the model

These models use a transformer-based causal decoder-only architecture. To speed up inference, they employ Grouped Query Attention with twelve query heads and four key-value heads. A high RoPE value of 1000042 helps the model manage long contexts effectively. Training involved 2 teratokens of data covering web content, code, science, technology, engineering and mathematics topics. The process used 1,024 H100 graphics processing units. After pretraining, the instruct version underwent post-training on 1.2 million samples focused on safety, function calls and conversational skills.

What it can do

The company states that Falcon 3 achieves top results at the time of release for reasoning and instruction following. It performs strongly in code and mathematics tasks. Evaluations on the Hugging Face Open LLM Leaderboard show competitive scores against similar sized models like Mistral-Nemo. The model supports multiple languages, though primarily European ones. Its design allows it to handle complex queries requiring long-term memory within the 32,000 token limit, making it suitable for detailed technical or academic work.

How to use it

Weights for the Falcon 3 family are available openly under the TII Falcon-LLM License 2.0. Developers can download the models directly from platforms like Hugging Face. The license permits use with certain conditions typical of open weight releases. Code examples are provided for loading the model using standard libraries such as Transformers. This approach allows researchers and engineers to run the models locally or deploy them on their own infrastructure without relying solely on an external application programming interface.

Safety and testing

Post-training included 1.2 million samples specifically curated for safety and alignment. The dataset covered conversational norms and function calling protocols to reduce harmful outputs. While specific red-teaming results are not detailed in the summary, the inclusion of safety data suggests a focus on responsible deployment. The model is instructed to follow user directions while maintaining helpful and friendly interactions, as indicated by the system prompt examples provided in the documentation.

Questions people ask about Falcon 3

What languages does Falcon 3 support?

Falcon 3 supports four languages: English, French, Spanish and Portuguese. It is designed to handle multilingual tasks within these specific European languages effectively.

How large is the context window?

The model supports a context length of up to 32,000 tokens. This allows it to process long documents or maintain coherence over extended conversations without losing earlier context.

Is Falcon 3 open source?

Yes, the weights are open and available under the TII Falcon-LLM License 2.0. Users can download and run the models locally, subject to the terms of this specific license.

What hardware was used for training?

The models were trained using 1,024 H100 GPU chips. This substantial compute resource allowed the team to process 2 teratokens of diverse data including web, code and STEM content.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Falcon 3 yet, so the facts on this page come from the sources listed above: huggingface.co/tiiuae/Falcon3-10B-Instruct. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.