News
ChinaQwen Incidents Show Self-Hosted AI Models Need Stronger SteeringIndiaSarvam sells DeepSeek on its Token Factory and Indian users are furiousFrontierGoogle’s Gemini 4 Neon extends its rapid run of model releasesIndiaIndiaAI Mission Shortlists 12 Teams for Indigenous AI Models: Minister Updates Rajya SabhaChinaSarvam Prices Its Own Model Below DeepSeek In New Token FactoryIndiaSarvam AI faces backlash over giving access to Chinese modelsIndiaSarvam defends serving foreign AI models on Token Factory, says it runs on privately procured GPUsIndiaSarvam AI’s LLM researcher catches ChatGPT using his laptop without authorisation — it apologisesIndiaSarvam sells DeepSeek access while pursuing sovereign AI modelsIndiaSarvam faces backlash for offering Chinese AI models on its platformChinaQwen Incidents Show Self-Hosted AI Models Need Stronger SteeringIndiaSarvam sells DeepSeek on its Token Factory and Indian users are furiousFrontierGoogle’s Gemini 4 Neon extends its rapid run of model releasesIndiaIndiaAI Mission Shortlists 12 Teams for Indigenous AI Models: Minister Updates Rajya SabhaChinaSarvam Prices Its Own Model Below DeepSeek In New Token FactoryIndiaSarvam AI faces backlash over giving access to Chinese modelsIndiaSarvam defends serving foreign AI models on Token Factory, says it runs on privately procured GPUsIndiaSarvam AI’s LLM researcher catches ChatGPT using his laptop without authorisation — it apologisesIndiaSarvam sells DeepSeek access while pursuing sovereign AI modelsIndiaSarvam faces backlash for offering Chinese AI models on its platform
EveryLLM
Eagle and Finch

Eagle and Finch

Efficient recurrent sequence models with matrix states

Eagle (RWKV-5) and Finch (RWKV-6) are sequence models that improve on the RWKV-4 architecture. Released in April 2024 by a large research team, they use matrix-valued states and dynamic recurrence to balance expressivity with inference speed. The models are open source under the Apache 2.0 licence.

Share on WhatsApp

  • Uses matrix-valued states for better expressivity
  • Trained on 1.12 trillion token multilingual corpus
  • Released under Apache 2.0 open licence
  • Maintains RNN-like inference efficiency
  • Includes fast tokenizer based on greedy matching

What it is

Eagle and Finch represent the fifth and sixth iterations of the RWKV family of sequence models. These systems aim to enhance the capabilities of their predecessor, RWKV-4, by introducing new architectural features. The primary goal is to maintain the fast inference speeds typical of recurrent neural networks while boosting the model's ability to understand complex patterns. The project includes four versions of Eagle and two of Finch, covering a range of sizes from less than half a billion to 7.5 billion parameters. All weights are available publicly.

How it came about

The research team, led by Bo Peng and including twenty-nine other contributors, submitted their findings in April 2024. The paper underwent several revisions, with the final version appearing in September of that year. This work builds directly on the earlier RWKV-4 architecture. The developers focused on creating models that could handle multilingual tasks more effectively, leading to the creation of a new training corpus and tokenizer specifically for this purpose.

Inside the model

The core innovation lies in the use of matrix-valued states within a multi-headed structure. This design allows the model to store more information at each step compared to previous vector-based approaches. Additionally, a dynamic recurrence mechanism adjusts how past information influences current predictions. To support these models, the team compiled a dataset containing 1.12 trillion tokens. They also developed a fast tokenizer that uses greedy matching to improve processing speed and multilingual support. The architecture retains the efficiency benefits of recurrent neural networks during inference.

What it can do

The models were tested across a broad set of benchmarks to evaluate their performance. The results indicate that both Eagle and Finch achieve competitive scores relative to other models in their size class. The inclusion of a large multilingual corpus suggests strong capabilities in handling diverse languages. The varying parameter counts allow users to select a model that fits their specific computational constraints, from small edge devices to larger servers.

How to use it

All models are released under the Apache 2.0 licence, which permits wide usage and modification. The weights can be downloaded from Hugging Face. The team also provides the code required for training, inference, and time-parallel training. This open approach ensures that researchers and developers can inspect, reproduce, and build upon the work without restrictive barriers.

Why it matters

By demonstrating that recurrent-style architectures can compete with transformer-based models in performance while offering better inference efficiency, this work contributes to the diversity of available AI structures. The release of full training and inference code supports further experimentation in the community. It highlights an alternative path for developing large language models that prioritises computational speed during deployment.

Questions people ask about Eagle and Finch

What is the main difference between Eagle and Finch?

Eagle refers to the RWKV-5 architecture, while Finch refers to RWKV-6. Both introduce matrix-valued states and dynamic recurrence, but they represent sequential improvements in the RWKV family. The paper presents four Eagle models and two Finch models of varying sizes.

Are these models open source?

Yes, all models are released on Hugging Face under the Apache 2.0 licence. This permissive licence allows for free use, modification, and distribution. The developers also provide the source code for training and inference.

How large are the Eagle and Finch models?

The Eagle models range in size from 0.46 billion to 7.5 billion parameters. The Finch models are available in two sizes: 1.6 billion and 3.1 billion parameters. This variety allows users to choose a model based on their hardware capabilities.

What makes these models efficient?

They are designed to maintain the inference efficiency characteristics of recurrent neural networks (RNNs). Unlike standard transformers that may require significant memory for long sequences, these models use dynamic recurrence and matrix-valued states to process data efficiently.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Eagle and Finch yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2404.05892. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.