ALIA-40b
Multilingual base model for 35 European languages
ALIA-40b is a decoder-only language model with 40 billion parameters, developed by Barcelona Supercomputing Center. Trained on 9.37 trillion tokens covering 35 European languages and code, it offers open weights under an Apache 2.0 licence for research and commercial use.
- Trained on 9.37 trillion tokens
- Supports 35 European languages
- Released under Apache 2.0 licence
- Built on MareNostrum 5 supercomputer
- 40 billion parameter decoder model
What it is
ALIA-40b is a base language model created by the Barcelona Supercomputing Center. It features 40 billion parameters and operates as a decoder-only transformer. The system was pre-trained from scratch on a massive corpus of 9.37 trillion tokens. Its primary focus is supporting 35 European languages alongside 92 programming languages. This version serves as a foundation for further fine-tuning rather than a ready-to-use assistant. Developers can use it for research or commercial projects after adding safety layers.
How it came about
The model was built using the NeMo Framework and PyTorch Lightning. Training took place on MareNostrum 5, a pre-exascale supercomputer located in Barcelona. The infrastructure included over a thousand nodes with Nvidia Hopper GPUs. The team released the base model along with its training scripts and configuration files. These resources are hosted publicly to support transparency and further development by the community.
Inside the model
The architecture uses SwiGLU activation functions and RMS Norm for layer normalisation. It employs grouped query attention to improve efficiency during processing. The training data came largely from Colossal OSCAR, which made up more than half the tokens. Other sources included Starcoder for code and FineWeb-Edu for educational content. The creators adjusted data proportions to boost representation of Spanish co-official languages like Catalan and Basque. English and code data were downsampled to balance this focus.
What it can do
As a base model, it generates text but lacks instruction tuning. This means it may produce unsafe or biased outputs if used directly. It supports generation in many European tongues and can handle code snippets. The model is designed to be adapted for specific tasks through post-training. Users must apply their own alignment steps before deploying it in high-stakes environments. It performs well on multilingual tasks due to its diverse training set.
How to use it
Weights are available under the permissive Apache 2.0 licence. This allows both academic research and commercial application. The developers provide full access to training scripts via GitHub. Inference can be run using standard tools like Hugging Face pipelines or vLLM. The open nature encourages modification and integration into larger systems without restrictive legal barriers.
Safety and testing
The release notes warn that this base model has not been aligned with human preferences. It may generate misleading or inappropriate content. The creators strongly recommend additional post-training stages to mitigate these risks. Deployment in production systems requires careful risk assessment. Malicious use or activities violating human rights are explicitly out of scope for intended usage.
Why it matters
By focusing on European languages, the model addresses gaps in multilingual AI coverage. It provides a strong baseline for regional language technologies. The open release of weights and training code supports reproducibility in the field. It enables other researchers to build specialised assistants for less-represented languages.
Questions people ask about ALIA-40b
Is ALIA-40b safe to use directly?
No, it is a base model without instruction tuning. It may produce unsafe or biased outputs. You must perform additional post-training and alignment before using it in production systems.
What languages does it support?
It covers 35 European languages, including Spanish, Catalan, Basque, and Galician. It also supports 92 programming languages, making it useful for code generation tasks alongside natural language processing.
Can I use it for commercial products?
Yes, the model is released under the Apache 2.0 licence. This permissive terms allows for both research and commercial use, provided you comply with applicable laws and regulations.
Good reading and sources
There is no English Wikipedia article on ALIA-40b yet, so the facts on this page come from the sources listed above: huggingface.co/BSC-LT/ALIA-40b. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.