News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
StarCoder

StarCoder

Open code generation model with 15.5 billion parameters

StarCoder is a 15.5 billion parameter language model for code, released in May 2023 by the BigCode community. It supports multiple programming languages and offers infilling capabilities, trained on one trillion tokens from permissively licensed GitHub repositories.

Share on WhatsApp

  • Contains 15.5 billion parameters
  • Trained on 1 trillion tokens from The Stack
  • Supports 8K context length
  • Achieves 40% pass@1 on HumanEval
  • Uses multi-query attention for speed

What it is

StarCoder is a large language model designed specifically for programming tasks. Developed by the BigCode community, it contains 15.5 billion parameters and supports an context window of 8,000 tokens. The model handles multiple programming languages and includes features for code infilling. It was built to provide a powerful, open-access alternative for developers seeking automated coding assistance.

How it came about

The BigCode community, an open scientific collaboration, introduced StarCoder in May 2023. The project aimed to foster responsible development of code-specific language models. The team released two versions: StarCoderBase, the foundational model, and StarCoder, which underwent additional fine-tuning. This release represented a significant step in making high-performance code models available to the public under open licences.

Inside the model

The architecture uses a decoder-only transformer design with multi-query attention, which enables faster inference during large-batch processing. StarCoderBase was trained on one trillion tokens sourced from The Stack, a collection of permissively licensed GitHub repositories. The final StarCoder model resulted from fine-tuning this base on 35 billion Python tokens. The training process included an improved pipeline for redacting personally identifiable information.

What it can do

Evaluations indicate that StarCoderBase outperforms other open code models supporting multiple languages. It matches or exceeds the performance of OpenAI’s code-cushman-001 model. The fine-tuned StarCoder variant excels in Python tasks, achieving a 40 per cent pass rate on the HumanEval benchmark while maintaining competence in other programming languages. It supports infilling, allowing it to generate code within existing structures.

How to use it

The models are publicly available under a version of the Open Responsible AI Model license, which the developers describe as commercially viable. Users can access the weights directly. The release includes tools for inspection and an opt-out process for data contributors. This approach aims to balance open access with responsible usage and legal compliance for commercial applications.

Safety and testing

The development team implemented several safety measures before release. These include an enhanced pipeline for redacting personally identifiable information from the training data. They also introduced a novel attribution tracing tool to help track the origins of generated code. These steps were taken to ensure a safer open-access model release and to address concerns about data privacy and copyright.

Why it matters

StarCoder provided a comprehensive baseline for evaluating code language models at the time of its release. By offering a high-performing open-weight model, it allowed researchers and developers to build upon a robust foundation without relying solely on proprietary APIs. Its use of The Stack dataset set a precedent for using permissively licensed code in large-scale training.

Questions people ask about StarCoder

What is StarCoder used for?

StarCoder is designed for code generation and completion across multiple programming languages. It supports infilling tasks and can assist developers by predicting code segments based on surrounding context.

Is StarCoder open source?

Yes, the model weights are publicly available under the Open Responsible AI Model license. This licence is described as more commercially viable than some previous open licences, allowing broader use.

How was StarCoder trained?

The base model was trained on one trillion tokens from The Stack, a dataset of permissively licensed GitHub repositories. It was then fine-tuned on 35 billion Python tokens to create the final StarCoder model.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on StarCoder yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2305.06161. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.