Poro
Open 34B parameter model for Finnish, English and code
Poro is a 34 billion parameter language model built by SiloGen, TurkuNLP and HPLT. Released in 2024, it trains on Finnish, English and code to support low-resource languages while maintaining strong general capabilities.
- 34 billion parameters trained on 1 trillion tokens
- Optimised for Finnish, English and programming code
- Built on LUMI supercomputer using AMD GPUs
- Released under Apache 2.0 open source licence
- Uses BLOOM architecture with ALiBi embeddings
What it is
Poro is a generative pretrained transformer with 34 billion parameters. It functions as a base model, meaning users must fine-tune it for specific tasks. The project aims to create open source tools for non-English and low-resource languages, specifically Finnish. By mixing English and Finnish data, the model achieves fluency in both, handles programming code, and performs basic translation between the two languages. It represents the first entry in a planned family of models.
How it came about
The model emerged from a partnership between SiloGen at Silo AI, the TurkuNLP group at the University of Turku, and High Performance Language Technologies. Work started in September 2023 on the LUMI supercomputer in Finland. The team released research checkpoints every 100 billion tokens during development. They published their findings in April 2024, arguing that multilingual training helps rather than hinders performance for smaller languages.
Inside the model
This decoder-only system follows the BLOOM architecture and uses ALiBi embeddings to handle context length during inference. Training utilised a custom fork of the Megatron-Deepspeed framework on 512 AMD MI250X GPUs. The dataset comprised one trillion tokens, mixing English, Finnish and code. Finnish sources included news archives, Wikipedia and social media. The tokenizer was custom-built with 128,000 entries to match the training mix.
What it can do
The model outperforms previous systems designed solely for Finnish. It remains competitive in English and coding tasks. Its primary strength lies in translation between Finnish and English. As a base model, it requires further adjustment for most applications. The developers note it has no meaningful skill in languages other than Finnish, English or code. Outputs may reflect biases present in the web data used for training.
How to use it
Weights, scripts and data are fully open under the Apache 2.0 licence. Users can download the model from Hugging Face. The repository includes branches for each training checkpoint, allowing researchers to inspect earlier stages of learning. The main branch always points to the final version. No API or commercial app is mentioned; distribution relies on direct access to the model files.
Safety and testing
The creators warn that the model may produce inaccurate or prejudiced content due to its training data. They advise users to exercise discretion and perform additional evaluations. The focus remains on openness and research utility rather than guarded deployment.
Reception and debate
The project positions itself as a proof that multilinguality benefits model training. It challenges the view that mixing languages dilutes performance. By releasing all components openly, it invites community scrutiny and adaptation. The paper highlights its success in advancing Finnish language processing capabilities beyond what monolingual models achieved.
Why it matters
Poro demonstrates that combining high-resource languages like English with low-resource ones like Finnish can yield superior results for the latter. It provides a template for other groups working on under-represented languages. The open release of checkpoints and code supports reproducibility and further research into efficient multilingual training strategies.
Questions people ask about Poro
What languages does Poro speak?
It is primarily optimized for Finnish and English, along with programming code. The developers state it has no meaningful proficiency in other languages. It can perform basic translation between Finnish and English.
Is Poro free to use?
Yes, the model weights, training scripts and data are released under the Apache 2.0 licence. This permits open use and modification. You can download it directly from Hugging Face.
Do I need to fine-tune Poro?
Yes, it is a base model. This means it predicts next tokens but lacks specific instruction-following abilities. You must fine-tune it for most practical use cases to align it with your needs.
Good reading and sources
There is no English Wikipedia article on Poro yet, so the facts on this page come from the sources listed above: huggingface.co/LumiOpen/Poro-34B, arxiv.org/abs/2404.01856. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.