InternLM2
Open-source 20 billion parameter language model with long context support
InternLM2 is an open-source large language model released in March 2024 by the InternLM team. It features a 20 billion parameter architecture designed for practical scenarios, offering strong performance in reasoning, coding, and long-context tasks. The model supports a 200,000 token context window and allows free commercial use under specific licence terms.
- Supports a 200,000 token context window
- 20 billion parameters in base and chat versions
- Allows free commercial usage with licence
- Uses Conditional Online RLHF for alignment
- Competes with GPT-3.5 in key benchmarks
What it is
InternLM2 is a large language model developed by the InternLM team. Released in March 2024, it includes a base model and a chat variant with 20 billion parameters. The system is designed for practical applications, emphasising improvements in reasoning, mathematics, and code generation. It supports a context window of 200,000 tokens, allowing it to process very long documents effectively. The model is available for both academic research and free commercial usage.
How it came about
The model represents the second generation of the InternLM series. Its development focused on overcoming challenges faced by previous open-source models in matching closed systems. The team published a technical report in March 2024 detailing the pre-training and optimisation processes. This release followed the original InternLM, aiming to provide deeper insights into model evolution by releasing weights from different training stages.
Inside the model
InternLM2 uses a decoder-only transformer architecture. Training involved diverse data types, including text, code, and specific long-context materials. The process began with a 4,000 token context length, expanding to 32,000 tokens during later pre-training and fine-tuning phases. Alignment was achieved through Supervised Fine-Tuning and a method called Conditional Online Reinforcement Learning from Human Feedback. This strategy aims to handle conflicting human preferences and prevent reward hacking.
What it can do
The model demonstrates strong performance in reasoning, math, and code tasks. Evaluations using OpenCompass show it competes with or exceeds GPT-3.5 in certain areas. With a code interpreter, it achieves results similar to GPT-4 on benchmarks like GSM8K and MATH. It excels in long-context tasks, such as finding specific information within large inputs. The system also supports complex agent building and multi-step tool calling for data analysis.
How to use it
Weights are openly available for academic research and free commercial use. Users must apply for a commercial licence via an online form if needed. The codebase is licensed under Apache-2.0. Deployment tools include LMDeploy and vLLM, which allow for local inference and OpenAI-compatible server setups. The model can be loaded using standard libraries like Transformers with remote code trust enabled.
Safety and testing
The developers implemented safety measures during training to encourage ethical and legal compliance. However, they note that the probabilistic nature of the model may still lead to unexpected outputs. These could include biases, discrimination, or harmful content. Users are advised not to propagate such material, and the developers disclaim responsibility for consequences arising from the spread of harmful information.
Questions people ask about InternLM2
What is the context length of InternLM2?
InternLM2 supports a context window of 200,000 tokens. This allows it to handle very long inputs and perform well on tasks requiring long-term dependency capture, such as the Needle-in-a-Haystack test.
Can I use InternLM2 for commercial purposes?
Yes, the model weights are open for academic research and allow free commercial usage. However, users must fill out an application form to obtain a commercial licence if required for their specific use case.
How does InternLM2 handle tool use?
The model has enhanced capabilities for instruction following, tool selection, and reflection. It supports the creation of complex agents and can perform multi-step tool calling, enabling tasks like data analysis and code interpretation.
Good reading and sources
There is no English Wikipedia article on InternLM2 yet, so the facts on this page come from the sources listed above: huggingface.co/internlm/internlm2-chat-20b, arxiv.org/abs/2403.17297. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.