News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
Bielik

Bielik

Polish-optimised 11 billion parameter language model

Bielik is an 11 billion parameter generative text model built by SpeakLeash and ACK Cyfronet AGH. Released in 2025, it uses depth up-scaling from Mistral architecture to process Polish text with high precision while retaining cross-lingual skills.

Share on WhatsApp

  • 11 billion parameters scaled from Mistral 7B architecture
  • Trained on over 10 billion tokens of Polish text
  • Outperforms 70B+ models on Polish language benchmarks
  • Uses weighted loss and adaptive learning rate techniques
  • Open weights with multiple quantisation options available

What it is

Bielik is a generative text model with 11 billion parameters, designed specifically for processing the Polish language. It was developed through a collaboration between the open-science project SpeakLeash and the High Performance Computing centre ACK Cyfronet AGH. The model aims to provide accurate responses and perform linguistic tasks with high precision for Polish users. It retains strong capabilities in other languages as well. The current version, Bielik-11B-v2.3-Instruct, is a linear merge of three earlier instruct-tuned variants. This approach combines their strengths to improve overall performance and reliability in chat and instruction-following scenarios.

How it came about

The model builds upon the Mistral 7B v0.2 architecture, which was scaled to 11 billion parameters using a technique called depth up-scaling. The development team included Krzysztof Ociepa, Łukasz Flis, Krzysztof Wróbel, Adrian Gwoździej, and Remigiusz Kinas. Training leveraged Polish large-scale computing infrastructure within the PLGrid environment, specifically the Athena and Helios supercomputers. A computational grant supported this work. The v2.3 release merged previous versions (v2.0, v2.1, and v2.2) to refine output quality. The technical report describing these innovations was published in May 2025.

Inside the model

Bielik uses a causal decoder-only transformer design. Training data consisted of over 20 million instructions totalling more than 10 billion tokens, curated from Polish corpora. To handle varying data quality, the team introduced Weighted Instruction Cross-Entropy Loss, which assigns weights based on example quality. They also used an Adaptive Learning Rate that adjusts according to context length. Alignment with user preferences employed Direct Preference Optimisation (DPO), specifically a variant called DPO-Positive. This method used both generated and manually corrected examples scored by a reward model. The training framework, ALLaMo, was implemented by Krzysztof Ociepa to efficiently train models with LLaMA-like architectures.

What it can do

The model demonstrates exceptional performance in Polish language benchmarks. On the Open PL LLM Leaderboard, it achieved an average score of 65.71, outperforming several larger models such as Meta-Llama-3-70B-Instruct and Mixtral-8x22B-Instruct. It handles tasks including sentiment analysis, categorisation, and text classification. The model also shows strong cross-lingual capabilities, allowing it to function effectively in English and other languages. Evaluations included the Polish EQ-Bench for emotional intelligence. Its parameter efficiency allows it to compete with models having two to six times more parameters.

How to use it

Bielik weights are available openly for download. The developers provide multiple quantised versions to support different hardware constraints, including GGUF formats with various precision levels and GPTQ 4-bit models. An FP8 version is optimised for specific GPU architectures like Ada Lovelace and Hopper. The model uses the ChatML prompt format for interactions. Users can run it locally using libraries such as transformers, vLLM, or SGLang. A public Chat Arena platform allows users to test and compare its performance against other models directly.

Why it matters

Bielik establishes new benchmarks for resource-efficient language modelling in less-represented languages. By demonstrating that an 11 billion parameter model can outperform much larger counterparts in specific linguistic contexts, it highlights the value of targeted training and architectural scaling. The project advances Polish language AI capabilities by providing a high-quality, open-weight alternative to proprietary systems. Its use of synthetic data generation and advanced loss functions offers a template for other regional language projects seeking to maximise performance with limited high-quality instruction data.

Questions people ask about Bielik

What is Bielik designed for?

Bielik is optimised for processing Polish text. It performs linguistic tasks such as sentiment analysis and text classification with high precision. It also maintains strong cross-lingual abilities for use in English and other languages.

How large is the Bielik model?

The model has 11 billion parameters. It was built by scaling up the Mistral 7B v0.2 architecture using depth up-scaling. This size allows it to run on consumer hardware when quantised.

Who developed Bielik?

It was created by SpeakLeash, an open-science project, in collaboration with ACK Cyfronet AGH, a high-performance computing centre in Poland. The work was supported by computational grants and local infrastructure.

Is Bielik free to use?

Yes, the model weights are open and available for download. Developers offer various quantised versions like GGUF and GPTQ to make it accessible for different hardware setups without licensing fees.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Bielik yet, so the facts on this page come from the sources listed above: huggingface.co/speakleash/Bielik-11B-v2.3-Instruct, arxiv.org/abs/2505.02410. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.