News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
fnInternLM2

InternLM2

Open-source 20 billion parameter language model with long context support

InternLM2 is an open-source large language model released in March 2024 by the InternLM team. It features a 20 billion parameter architecture designed for practical scenarios, offering strong performance in reasoning, coding, and long-context tasks. The model supports a 200,000 token context window and allows free commercial use under specific licence terms.

Share on WhatsApp

  • Supports a 200,000 token context window
  • 20 billion parameters in base and chat versions
  • Allows free commercial usage with licence
  • Uses Conditional Online RLHF for alignment
  • Competes with GPT-3.5 in key benchmarks

What it is

InternLM2 is a large language model developed by the InternLM team. Released in March 2024, it includes a base model and a chat variant with 20 billion parameters. The system is designed for practical applications, emphasising improvements in reasoning, mathematics, and code generation. It supports a context window of 200,000 tokens, allowing it to process very long documents effectively. The model is available for both academic research and free commercial usage.

How it came about

The model represents the second generation of the InternLM series. Its development focused on overcoming challenges faced by previous open-source models in matching closed systems. The team published a technical report in March 2024 detailing the pre-training and optimisation processes. This release followed the original InternLM, aiming to provide deeper insights into model evolution by releasing weights from different training stages.

Inside the model

InternLM2 uses a decoder-only transformer architecture. Training involved diverse data types, including text, code, and specific long-context materials. The process began with a 4,000 token context length, expanding to 32,000 tokens during later pre-training and fine-tuning phases. Alignment was achieved through Supervised Fine-Tuning and a method called Conditional Online Reinforcement Learning from Human Feedback. This strategy aims to handle conflicting human preferences and prevent reward hacking.

What it can do

The model demonstrates strong performance in reasoning, math, and code tasks. Evaluations using OpenCompass show it competes with or exceeds GPT-3.5 in certain areas. With a code interpreter, it achieves results similar to GPT-4 on benchmarks like GSM8K and MATH. It excels in long-context tasks, such as finding specific information within large inputs. The system also supports complex agent building and multi-step tool calling for data analysis.

How to use it

Weights are openly available for academic research and free commercial use. Users must apply for a commercial licence via an online form if needed. The codebase is licensed under Apache-2.0. Deployment tools include LMDeploy and vLLM, which allow for local inference and OpenAI-compatible server setups. The model can be loaded using standard libraries like Transformers with remote code trust enabled.

Safety and testing

The developers implemented safety measures during training to encourage ethical and legal compliance. However, they note that the probabilistic nature of the model may still lead to unexpected outputs. These could include biases, discrimination, or harmful content. Users are advised not to propagate such material, and the developers disclaim responsibility for consequences arising from the spread of harmful information.

Questions people ask about InternLM2

What is the context length of InternLM2?

InternLM2 supports a context window of 200,000 tokens. This allows it to handle very long inputs and perform well on tasks requiring long-term dependency capture, such as the Needle-in-a-Haystack test.

Can I use InternLM2 for commercial purposes?

Yes, the model weights are open for academic research and allow free commercial usage. However, users must fill out an application form to obtain a commercial licence if required for their specific use case.

How does InternLM2 handle tool use?

The model has enhanced capabilities for instruction following, tool selection, and reflection. It supports the creation of complex agents and can perform multi-step tool calling, enabling tasks like data analysis and code interpretation.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on InternLM2 yet, so the facts on this page come from the sources listed above: huggingface.co/internlm/internlm2-chat-20b, arxiv.org/abs/2403.17297. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.