News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
Falcon2-11B

Falcon2-11B

An 11 billion parameter causal decoder model by TII.

TII built Falcon2-11B, an 11 billion parameter causal decoder-only language model. It was trained on over five trillion tokens using a four-stage strategy to extend context length. The model supports English and several European languages under the TII Falcon License 2.0.

Share on WhatsApp

  • Trained on over 5,000 billion tokens
  • Supports 11 European languages
  • Context length extends to 8192 tokens
  • Uses FlashAttention-2 and multiquery attention
  • Released under TII Falcon License 2.0

What it is

Falcon2-11B is a large language model with 11 billion parameters. It uses a causal decoder-only architecture, meaning it predicts the next token in a sequence. The Technology Innovation Institute (TII) developed this model. It is designed for research and as a foundation for further specialisation. Users can fine-tune it for tasks like summarisation or chatbots. The model handles English and ten other European languages.

How it came about

TII released Falcon2-11B in 2024. The team published a technical report detailing its creation. This model follows earlier versions in the Falcon family. It was built to improve upon previous iterations by enhancing data quality and context handling. The development involved a specific four-stage training strategy to manage increasing complexity and context length effectively.

Inside the model

The model employs a causal decoder-only structure adapted from GPT-3 designs. It uses rotary positional embeddings and multiquery attention mechanisms. FlashAttention-2 optimises the attention process. Training occurred in four stages. The first three stages increased the context window from 2048 to 8192 tokens. The final stage focused on high-quality data. TII used 1024 A100 40GB GPUs on AWS SageMaker. The custom Gigatron codebase managed distributed training with 3D parallelism.

What it can do

Falcon2-11B processes English, German, Spanish, French, Italian, Portuguese, Polish, Dutch, Romanian, Czech, and Swedish. It may not generalise well to other languages. Benchmarks show an ARC-Challenge score of 59.73 with 25 shots and 50.17 with zero shots. As a raw pretrained model, it requires fine-tuning for most specific applications. It carries biases present in its web-based training data.

How to use it

The weights are available under the TII Falcon License 2.0. This licence is based on Apache 2.0 but includes an acceptable use policy. Users must agree to responsible AI practices. The model is hosted on Hugging Face. Developers can use it for inference, fine-tuning, and quantisation. PyTorch 2.0 is required for use with the transformers library.

Safety and testing

TII notes that the model reflects stereotypes found in online data. It recommends implementing guardrails for production use. The licence includes an acceptable use policy to promote responsible deployment. Users should assess risks before deploying the model in live environments. Fine-tuning is advised to mitigate specific biases for targeted tasks.

Questions people ask about Falcon2-11B

What languages does Falcon2-11B support?

It supports English, German, Spanish, French, Italian, Portuguese, Polish, Dutch, Romanian, Czech, and Swedish. It is not designed to generalise well to other languages outside this set.

Is Falcon2-11B ready for production use?

No, it is a raw pretrained model. TII recommends fine-tuning it for specific use cases and adding guardrails. Production use requires adequate risk assessment and mitigation strategies.

What hardware was used to train the model?

Training used 1024 A100 40GB GPUs on AWS SageMaker. The process took roughly two months using a custom distributed training codebase called Gigatron with 3D parallelism.

What is the licence for Falcon2-11B?

It uses the TII Falcon License 2.0. This is a permissive licence based on Apache 2.0. It includes an acceptable use policy that requires users to promote responsible AI usage.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Falcon2-11B yet, so the facts on this page come from the sources listed above: huggingface.co/tiiuae/falcon-11B. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.