News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
OpenCoder

OpenCoder

Open code model family with full training data and pipeline

OpenCoder is a family of open code large language models released in November 2024 by a research team. It includes 1.5 billion and 8 billion parameter versions for English and Chinese. The project releases weights, training data, and processing pipelines to support reproducible scientific research in code AI.

Share on WhatsApp

  • Pretrained on 2.5 trillion tokens of code and web data
  • Releases full training data and processing pipelines
  • Supports English and Chinese languages
  • 8B model scores 83.5 on HumanEval benchmark
  • Permissive licence allows commercial use

What it is

OpenCoder is a family of large language models designed specifically for coding tasks. Released in November 2024, it offers base and instruction-tuned variants with 1.5 billion and 8 billion parameters. The models support both English and Chinese languages. Unlike many proprietary systems, this project provides full transparency by releasing not just the model weights but also the complete training data and processing pipelines. This approach aims to serve as an open foundation for researchers studying code artificial intelligence.

How it came about

The model was introduced in a paper submitted to arXiv in November 2024, with revisions continuing into March 2025. The development team focused on creating a reproducible framework for building high-performance code models. They aimed to address the scarcity of open resources suitable for rigorous scientific investigation. By sharing every step of their process, from data cleaning to final training protocols, they sought to accelerate advancements in the field through community collaboration and transparent methods.

Inside the model

The models were pretrained from scratch on 2.5 trillion tokens. This dataset consisted of ninety percent raw code and ten percent code-related web text. The team applied heuristic rules for data cleaning and deduplication at both file and repository levels. They also incorporated high-quality synthetic data during the annealing and supervised fine-tuning stages. The instruction-tuned versions were trained on over 4.5 million examples. The architecture supports context windows of 4,000 tokens for the smaller model and 8,000 for the larger one.

What it can do

OpenCoder achieves performance comparable to leading proprietary models in code generation and reasoning. On the HumanEval benchmark, the 8 billion parameter instruct model scores 83.5, while the 1.5 billion version scores 72.5. For MBPP, the larger model reaches 79.1 compared to 72.7 for the smaller one. It also shows results on BigCodeBench-Hard, LiveCodeBench, and MultiPL-E. These scores position it among the top open-source options for coding tasks across multiple evaluation suites.

How to use it

The model weights and inference code are available via Hugging Face. The project operates under a permissive licence that allows for commercial applications. Beyond the models themselves, the team releases the complete data-cleaning code, synthetic data generation processes, and extensive checkpoints. This comprehensive open-source approach ensures that other developers and researchers can reproduce the training pipeline and build upon the provided foundation without restrictive barriers.

Why it matters

By providing a complete cookbook for building top-tier code models, OpenCoder influences how researchers approach open-source artificial intelligence. It demonstrates that high performance can be achieved with transparent data processing and reproducible methods. The release of detailed ablation studies and training protocols helps the community understand the key ingredients for success in code AI, potentially lowering the resource barriers for future scientific investigations in this domain.

Questions people ask about OpenCoder

What sizes are available for OpenCoder?

The family includes two main sizes: 1.5 billion parameters and 8 billion parameters. Each size has both a base model and an instruction-tuned variant, offering flexibility for different computational needs and use cases.

Is OpenCoder free to use commercially?

Yes, the OpenCoder series supports commercial applications under a permissive licence. This allows businesses and developers to integrate the models into their products without facing restrictive legal barriers typical of some other open-weight models.

What makes OpenCoder different from other code models?

It releases not only the model weights but also the complete training data, data cleaning code, and processing pipelines. This level of transparency allows for fully reproducible research, which is rare among high-performing code large language models.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on OpenCoder yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2411.04905, huggingface.co/infly/OpenCoder-8B-Instruct. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.