DeepSeek-V2
Efficient mixture-of-experts language model with 236 billion parameters
DeepSeek-V2 is a large language model built by DeepSeek-AI, released in May 2024. It uses a mixture-of-experts design to activate only 21 billion of its 236 billion parameters per token, aiming for lower training costs and faster inference while supporting long contexts.
- Contains 236 billion total parameters
- Activates only 21 billion parameters per token
- Supports context length of 128,000 tokens
- Trained on 8.1 trillion tokens
- Uses Multi-head Latent Attention for efficiency
What it is
DeepSeek-V2 is a general-purpose language model developed by DeepSeek-AI. Announced in May 2024, it contains 236 billion parameters but activates only 21 billion for each token processed. This sparse activation strategy allows the model to handle complex tasks while keeping computational demands manageable. It supports a context window of 128,000 tokens, enabling it to process very long documents or conversations. The model is designed for both English and Chinese tasks, including coding and mathematical reasoning.
How it came about
The model was introduced by DeepSeek-AI in May 2024 as a successor to the dense DeepSeek 67B model. The development team focused on creating a more economical training process and efficient inference mechanism. By shifting from a dense architecture to a mixture-of-experts design, the creators aimed to improve performance while reducing costs. The release included both a base model and chat versions refined through supervised fine-tuning and reinforcement learning.
Inside the model
DeepSeek-V2 uses a mixture-of-experts architecture called DeepSeekMoE, which enables sparse computation. For attention mechanisms, it employs Multi-head Latent Attention (MLA). This technique compresses the key-value cache into a latent vector, significantly reducing memory usage during inference. The model was pretrained on 8.1 trillion tokens from diverse sources. After pretraining, it underwent supervised fine-tuning and reinforcement learning to align its outputs with user instructions and improve conversational abilities.
What it can do
The model performs strongly on benchmarks for English and Chinese. On MMLU, it scores 78.5, and on C-Eval, it reaches 81.7. In coding tasks, it achieves 48.8 on HumanEval and 66.6 on MBPP. Mathematical reasoning scores include 79.2 on GSM8K. The chat version, refined with reinforcement learning, shows competitive results in open-ended generation evaluations such as AlpacaEval 2.0 and Alignbench. It maintains performance across its full 128K context window.
How to use it
DeepSeek-AI provides the model weights openly via Hugging Face. Users can download the base model and chat variants for local execution. Running the model in BF16 format requires substantial hardware, specifically eight GPUs with 80GB memory each. The company also offers an API platform compatible with OpenAI standards, allowing developers to access the model via cloud services with pay-as-you-go pricing.
Questions people ask about DeepSeek-V2
How many parameters does DeepSeek-V2 have?
The model has 236 billion total parameters. However, it uses a mixture-of-experts architecture that activates only 21 billion parameters for each token processed, making it more efficient than dense models of similar size.
What is the context window size?
DeepSeek-V2 supports a context length of 128,000 tokens. This allows it to process and generate text based on very long inputs, such as extensive documents or lengthy conversation histories, without losing track of earlier information.
Is DeepSeek-V2 available for free?
Yes, the model weights are open and available for download on Hugging Face. Developers can run it locally if they have sufficient GPU memory. An API is also available through the DeepSeek platform, offering free tokens and paid usage options.
Good reading and sources
There is no English Wikipedia article on DeepSeek-V2 yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2405.04434, huggingface.co/deepseek-ai/DeepSeek-V2. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.