News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
Kimi Linear

Kimi Linear

Hybrid linear attention model with 48 billion parameters

Kimi Linear is a hybrid language model released by the Kimi Team in October 2025. It uses 48 billion total parameters with only 3 billion active per token to improve speed and reduce memory use for long contexts.

Share on WhatsApp

  • 48 billion total parameters with 3 billion active
  • Supports context lengths up to 1 million tokens
  • Reduces KV cache usage by up to 75 percent
  • Trained on 5.7 trillion tokens of data
  • Open weights available on Hugging Face

What it is

Kimi Linear is a language model architecture designed for high efficiency and performance across short and long contexts. Developed by the Kimi Team at Moonshot AI, it was released in October 2025. The model contains 48 billion total parameters but activates only 3 billion during inference. This design allows it to handle context windows up to one million tokens while significantly reducing memory usage compared to traditional full attention models.

How it came about

The Kimi Team introduced this architecture in a paper submitted to arXiv in late October 2025. The work builds on previous research into gated delta networks and linear attention mechanisms. The team released both base and instruction-tuned checkpoints trained on 5.7 trillion tokens. They also open-sourced the core kernel implementations to support further research and deployment in frameworks like vLLM.

Inside the model

The model uses a hybrid structure that combines Kimi Delta Attention (KDA) with Multi-Head Latent Attention (MLA) in a three-to-one ratio. KDA refines the gated delta rule with finer gating to better use finite-state recurrent memory. This approach reduces the need for large key-value caches by up to 75 percent. The architecture employs a chunkwise algorithm with specialized transition matrices to boost hardware efficiency and decoding throughput.

What it can do

Tests show the model achieves Pareto-optimal performance on long-context benchmarks like RULER. It records a score of 84.3 on RULER with a 128k context length. On MMLU-Pro with a 4k context, it scores 51.0. The system offers up to 6.3 times faster time per output token compared to standard MLA designs. It maintains competitive speed with full attention methods while handling sequences as long as one million tokens.

How to use it

Moonshot AI has made the model weights available under an open license. Developers can download the base and instruct versions from Hugging Face. The team provides code for integrating the model with Hugging Face Transformers and vLLM. This allows users to run local inference or create API endpoints compatible with OpenAI standards.

Questions people ask about Kimi Linear

How many parameters does Kimi Linear have?

The model has 48 billion total parameters. However, it is a mixture of experts style design where only 3 billion parameters are activated for each token during processing.

What is the maximum context length?

Kimi Linear supports a context window of up to one million tokens. It is designed to maintain efficiency and performance even at these extreme sequence lengths.

Is the model open source?

Yes, the team has released the model weights openly. You can download the base and instruction-tuned versions from Hugging Face and use them with standard libraries.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Kimi Linear yet, so the facts on this page come from the sources listed above: huggingface.co/moonshotai/Kimi-Linear-48B-A3B-Instruct, arxiv.org/abs/2510.26692. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.