Bielik
Polish-optimised 11 billion parameter language model
Bielik is an 11 billion parameter generative text model built by SpeakLeash and ACK Cyfronet AGH. Released in 2025, it uses depth up-scaling from Mistral architecture to process Polish text with high precision while retaining cross-lingual skills.
- 11 billion parameters scaled from Mistral 7B architecture
- Trained on over 10 billion tokens of Polish text
- Outperforms 70B+ models on Polish language benchmarks
- Uses weighted loss and adaptive learning rate techniques
- Open weights with multiple quantisation options available
What it is
Bielik is a generative text model with 11 billion parameters, designed specifically for processing the Polish language. It was developed through a collaboration between the open-science project SpeakLeash and the High Performance Computing centre ACK Cyfronet AGH. The model aims to provide accurate responses and perform linguistic tasks with high precision for Polish users. It retains strong capabilities in other languages as well. The current version, Bielik-11B-v2.3-Instruct, is a linear merge of three earlier instruct-tuned variants. This approach combines their strengths to improve overall performance and reliability in chat and instruction-following scenarios.
How it came about
The model builds upon the Mistral 7B v0.2 architecture, which was scaled to 11 billion parameters using a technique called depth up-scaling. The development team included Krzysztof Ociepa, Łukasz Flis, Krzysztof Wróbel, Adrian Gwoździej, and Remigiusz Kinas. Training leveraged Polish large-scale computing infrastructure within the PLGrid environment, specifically the Athena and Helios supercomputers. A computational grant supported this work. The v2.3 release merged previous versions (v2.0, v2.1, and v2.2) to refine output quality. The technical report describing these innovations was published in May 2025.
Inside the model
Bielik uses a causal decoder-only transformer design. Training data consisted of over 20 million instructions totalling more than 10 billion tokens, curated from Polish corpora. To handle varying data quality, the team introduced Weighted Instruction Cross-Entropy Loss, which assigns weights based on example quality. They also used an Adaptive Learning Rate that adjusts according to context length. Alignment with user preferences employed Direct Preference Optimisation (DPO), specifically a variant called DPO-Positive. This method used both generated and manually corrected examples scored by a reward model. The training framework, ALLaMo, was implemented by Krzysztof Ociepa to efficiently train models with LLaMA-like architectures.
What it can do
The model demonstrates exceptional performance in Polish language benchmarks. On the Open PL LLM Leaderboard, it achieved an average score of 65.71, outperforming several larger models such as Meta-Llama-3-70B-Instruct and Mixtral-8x22B-Instruct. It handles tasks including sentiment analysis, categorisation, and text classification. The model also shows strong cross-lingual capabilities, allowing it to function effectively in English and other languages. Evaluations included the Polish EQ-Bench for emotional intelligence. Its parameter efficiency allows it to compete with models having two to six times more parameters.
How to use it
Bielik weights are available openly for download. The developers provide multiple quantised versions to support different hardware constraints, including GGUF formats with various precision levels and GPTQ 4-bit models. An FP8 version is optimised for specific GPU architectures like Ada Lovelace and Hopper. The model uses the ChatML prompt format for interactions. Users can run it locally using libraries such as transformers, vLLM, or SGLang. A public Chat Arena platform allows users to test and compare its performance against other models directly.
Why it matters
Bielik establishes new benchmarks for resource-efficient language modelling in less-represented languages. By demonstrating that an 11 billion parameter model can outperform much larger counterparts in specific linguistic contexts, it highlights the value of targeted training and architectural scaling. The project advances Polish language AI capabilities by providing a high-quality, open-weight alternative to proprietary systems. Its use of synthetic data generation and advanced loss functions offers a template for other regional language projects seeking to maximise performance with limited high-quality instruction data.
Questions people ask about Bielik
What is Bielik designed for?
Bielik is optimised for processing Polish text. It performs linguistic tasks such as sentiment analysis and text classification with high precision. It also maintains strong cross-lingual abilities for use in English and other languages.
How large is the Bielik model?
The model has 11 billion parameters. It was built by scaling up the Mistral 7B v0.2 architecture using depth up-scaling. This size allows it to run on consumer hardware when quantised.
Who developed Bielik?
It was created by SpeakLeash, an open-science project, in collaboration with ACK Cyfronet AGH, a high-performance computing centre in Poland. The work was supported by computational grants and local infrastructure.
Is Bielik free to use?
Yes, the model weights are open and available for download. Developers offer various quantised versions like GGUF and GPTQ to make it accessible for different hardware setups without licensing fees.
Good reading and sources
There is no English Wikipedia article on Bielik yet, so the facts on this page come from the sources listed above: huggingface.co/speakleash/Bielik-11B-v2.3-Instruct, arxiv.org/abs/2505.02410. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.