PLLuM
Polish-focused large language model family
PLLuM is a suite of large language models built for Polish and Slavic languages. Developed by CYFRAGOVPL, it uses up to 150 billion tokens of text. The models support general tasks and public administration workflows.
- Trained on up to 150 billion Polish tokens
- Includes 70 billion parameter dense models
- Uses manual organic instruction datasets
- Top scores in Polish admin benchmarks
- Supports Slavic and Baltic languages
What it is
PLLuM is a collection of large language models designed primarily for Polish, with support for other Slavic and Baltic tongues. CYFRAGOVPL created the family to handle general text generation and specific administrative tasks. The series includes dense models with 8, 12, and 70 billion parameters, as well as mixture of experts variants. Training data reaches 150 billion tokens, ensuring strong linguistic coverage. These systems aim to provide coherent answers and serve as bases for specialized assistants.
How it came about
The project emerged from collaboration with various data providers to gather high-quality text. Developers focused on creating a robust Polish corpus, cleaning and deduplicating around 150 billion tokens. They released multiple versions in late 2024, denoted by the suffix 2412. Earlier naming conventions included PLLuM-12B-instruct before standardizing the family structure. The team prioritized manual curation over automated distillation to preserve linguistic nuance.
Inside the model
Models rely on decoder-only transformer architectures, some based on Llama 3.1 and others on Mistral or Mixtral backbones. Pretraining used vast Polish datasets, with a subset of 28 billion tokens available for fully open commercial use. Instruction tuning involved 40,000 manually written prompts and 50,000 converted instructions. A unique preference corpus helped align outputs for safety and balance. This human-led approach aimed to reduce negative transfer from non-Polish pretraining data.
What it can do
The models excel in Polish language tasks, achieving top scores on custom benchmarks for public administration. They handle question answering, summarization, and complex information retrieval effectively. Performance remains strong across general Slavic linguistic challenges. The chat variants are optimized for dialogue, offering safer responses in adversarial contexts. Domain-specific adaptations use retrieval augmented generation to improve accuracy in official settings.
How to use it
Weights are available via Hugging Face under mixed licensing terms. Some variants use Apache 2.0 or Llama 3.1 licenses, permitting broader use. Others carry CC-BY-NC-4.0 restrictions for non-commercial applications only. The 8x7B mixture of experts models offer efficient inference. Users can select base, instruct, or chat versions depending on their alignment needs and legal constraints.
Safety and testing
Developers created the first Polish preference corpus to teach balanced responses. Annotators from diverse backgrounds assessed model outputs for factual correctness and tone. This process targeted controversial topics to mitigate risks. The chat models underwent specific alignment to ensure efficiency and safety in general-purpose dialogues. Manual assessment helped address subtle linguistic issues that automated methods might miss.
Questions people ask about PLLuM
What languages does PLLuM support?
It focuses on Polish but also covers other Slavic and Baltic languages. English data was included to help with generalization. The models are designed to handle multilingual tasks within this regional scope.
Can I use PLLuM for commercial projects?
Some models allow commercial use under Apache 2.0 or Llama 3.1 licenses. Others use CC-BY-NC-4.0, which restricts usage to non-commercial purposes. Check the specific license for each variant before deployment.
How was the training data collected?
CYFRAGOVPL gathered large-scale text corpora, cleaning and deduplicating around 150 billion tokens. They also created manual instruction sets and preference data to refine model behavior and ensure linguistic quality.
Good reading and sources
There is no English Wikipedia article on PLLuM yet, so the facts on this page come from the sources listed above: huggingface.co/CYFRAGOVPL/PLLuM-12B-instruct. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.