News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
Falcon-H1

Falcon-H1

Hybrid transformer and state-space model series by TII

Falcon-H1 is a family of large language models developed by the Technology Innovation Institute in the United Arab Emirates. Released in July 2025, it uses a hybrid architecture combining transformers with state space models to improve efficiency. The series ranges from 0.5 billion to 34 billion parameters and supports long context windows.

Share on WhatsApp

  • Uses hybrid transformer and state space model architecture
  • Supports context windows up to 256,000 tokens
  • Flagship 34B model rivals 70B scale competitors
  • Released under permissive Falcon-LLM License
  • Available in sizes from 0.5B to 34B parameters

What it is

Falcon-H1 is a series of large language models created by the Technology Innovation Institute. It launched in July 2025 with sizes ranging from 0.5 billion to 34 billion parameters. The models use a hybrid design that merges standard transformer attention with state space models. This approach aims to balance high performance with computational efficiency. The series includes both base and instruction-tuned versions for various tasks.

How it came about

The Technology Innovation Institute developed this family to challenge conventional model designs. Earlier Falcon models relied solely on transformer or Mamba architectures. Falcon-H1 introduces a parallel hybrid approach. The team released over 30 checkpoints on the Hugging Face Hub. These include quantized instruction-tuned variants. The technical report details the systematic revisiting of data strategy and training dynamics.

Inside the model

The architecture combines transformer-based attention with state space models. State space models offer better memory for long contexts and improved efficiency. The models support context windows up to 256,000 tokens. They are trained to handle 18 languages. The design focuses on parameter and training efficiency. The flagship 34 billion parameter model matches larger competitors using less data.

What it can do

The models perform well in reasoning, mathematics, and scientific knowledge tasks. The 34 billion parameter version competes with models up to 70 billion parameters. It scores highly on benchmarks like MMLU and GSM8k. Smaller variants also show strong results. The 0.5 billion model performs comparably to typical 7 billion models from 2024. Instruction following and multilingual capabilities are key strengths.

How to use it

All models are available under the permissive Falcon-LLM License. Users can access them via the Hugging Face Hub. Inference is supported through libraries such as transformers, vLLM, and llama.cpp. The release includes base and instruction-tuned checkpoints. Quantized versions are also provided for easier deployment.

Questions people ask about Falcon-H1

What makes Falcon-H1 different from previous models?

It uses a parallel hybrid architecture that combines transformer attention with state space models. This design improves efficiency and long-context memory compared to pure transformer or Mamba models.

How large are the Falcon-H1 models?

The series includes configurations with 0.5 billion, 1.5 billion, 3 billion, 7 billion, and 34 billion parameters. Both base and instruction-tuned variants are available.

Can I use Falcon-H1 commercially?

Yes, the models are released under the Falcon-LLM License, which is described as a permissive open-source license. This allows for broad usage and accessibility.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Falcon-H1 yet, so the facts on this page come from the sources listed above: huggingface.co/tiiuae/Falcon-H1-34B-Instruct, arxiv.org/abs/2507.22448. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.