News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
DeepSeek-V2

DeepSeek-V2

Efficient mixture-of-experts language model with 236 billion parameters

DeepSeek-V2 is a large language model built by DeepSeek-AI, released in May 2024. It uses a mixture-of-experts design to activate only 21 billion of its 236 billion parameters per token, aiming for lower training costs and faster inference while supporting long contexts.

Share on WhatsApp

  • Contains 236 billion total parameters
  • Activates only 21 billion parameters per token
  • Supports context length of 128,000 tokens
  • Trained on 8.1 trillion tokens
  • Uses Multi-head Latent Attention for efficiency

What it is

DeepSeek-V2 is a general-purpose language model developed by DeepSeek-AI. Announced in May 2024, it contains 236 billion parameters but activates only 21 billion for each token processed. This sparse activation strategy allows the model to handle complex tasks while keeping computational demands manageable. It supports a context window of 128,000 tokens, enabling it to process very long documents or conversations. The model is designed for both English and Chinese tasks, including coding and mathematical reasoning.

How it came about

The model was introduced by DeepSeek-AI in May 2024 as a successor to the dense DeepSeek 67B model. The development team focused on creating a more economical training process and efficient inference mechanism. By shifting from a dense architecture to a mixture-of-experts design, the creators aimed to improve performance while reducing costs. The release included both a base model and chat versions refined through supervised fine-tuning and reinforcement learning.

Inside the model

DeepSeek-V2 uses a mixture-of-experts architecture called DeepSeekMoE, which enables sparse computation. For attention mechanisms, it employs Multi-head Latent Attention (MLA). This technique compresses the key-value cache into a latent vector, significantly reducing memory usage during inference. The model was pretrained on 8.1 trillion tokens from diverse sources. After pretraining, it underwent supervised fine-tuning and reinforcement learning to align its outputs with user instructions and improve conversational abilities.

What it can do

The model performs strongly on benchmarks for English and Chinese. On MMLU, it scores 78.5, and on C-Eval, it reaches 81.7. In coding tasks, it achieves 48.8 on HumanEval and 66.6 on MBPP. Mathematical reasoning scores include 79.2 on GSM8K. The chat version, refined with reinforcement learning, shows competitive results in open-ended generation evaluations such as AlpacaEval 2.0 and Alignbench. It maintains performance across its full 128K context window.

How to use it

DeepSeek-AI provides the model weights openly via Hugging Face. Users can download the base model and chat variants for local execution. Running the model in BF16 format requires substantial hardware, specifically eight GPUs with 80GB memory each. The company also offers an API platform compatible with OpenAI standards, allowing developers to access the model via cloud services with pay-as-you-go pricing.

Questions people ask about DeepSeek-V2

How many parameters does DeepSeek-V2 have?

The model has 236 billion total parameters. However, it uses a mixture-of-experts architecture that activates only 21 billion parameters for each token processed, making it more efficient than dense models of similar size.

What is the context window size?

DeepSeek-V2 supports a context length of 128,000 tokens. This allows it to process and generate text based on very long inputs, such as extensive documents or lengthy conversation histories, without losing track of earlier information.

Is DeepSeek-V2 available for free?

Yes, the model weights are open and available for download on Hugging Face. Developers can run it locally if they have sufficient GPU memory. An API is also available through the DeepSeek platform, offering free tokens and paid usage options.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on DeepSeek-V2 yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2405.04434, huggingface.co/deepseek-ai/DeepSeek-V2. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.