ALLaM
Bilingual Arabic and English language model series
ALLaM is a series of large language models built by the National Center for Artificial Intelligence at SDAIA in Saudi Arabia. Released in 2024, it focuses on advancing Arabic language technology while maintaining strong English capabilities through mixed-language training.
- Trained on 5.2 trillion mixed tokens
- Developed by Saudi NCAI at SDAIA
- Supports Arabic and English languages
- Uses decoder-only transformer architecture
- 7 billion parameters in preview release
What it is
ALLaM is a family of large language models designed to support Arabic Language Technology. The National Center for Artificial Intelligence at the Saudi Data and AI Authority developed the series. It operates as an autoregressive transformer that processes both Arabic and English text. The initial public release features a 7 billion parameter instruction-tuned variant. This model aims to serve as a foundational component for developers building AI systems in the region.
How it came about
The project was detailed in a technical paper published in July 2024. Researchers led by M Saiful Bari and colleagues outlined the training methodology. The team focused on transferring knowledge from English to Arabic without losing original capabilities. They released preview weights on Hugging Face to facilitate experimental initiatives and product development within the Arabic tech ecosystem.
Inside the model
The architecture uses a decoder-only transformer design. Training involved two main stages: first on 4 trillion English tokens, then on 1.2 trillion mixed Arabic and English tokens. This approach prevents catastrophic forgetting of English while acquiring Arabic proficiency. The team used parallel and translated data to align knowledge between languages. Human preference alignment was applied using 7 million instructions and 260,000 preference pairs. The codebase relies on NVIDIA Megatron-LM with bf16-mixed precision.
What it can do
The model performs well on various Arabic benchmarks. It achieves strong results on Arabic MMLU, ACVA, and Arabic Exams datasets. Evaluation also included local tests such as the General Aptitude Test and questions from the Saudi Ministry of Education. The system handles multiple-choice questions across humanities, STEM, and Islamic studies. It supports custom system prompts but functions effectively without predefined instructions.
How to use it
Weights for the 7 billion parameter instruct preview are available on Hugging Face. Developers can load the model using standard transformer libraries. The license terms are specified in the accompanying file. The model is intended for integration into larger AI systems, requiring developers to implement their own safety measures.
Safety and testing
The developers note that generative models have inherent uncertainties. Outputs may occasionally be incorrect or biased. They advise thorough safety evaluations and specific adjustments for intended use cases. The generated content does not represent official statements from NCAI or SDAIA. Risks from integrating with external tools must be minimized by users.
Why it matters
ALLaM contributes to the ecosystem of Arabic Language Technologies. It demonstrates how second-language acquisition via vocabulary expansion can steer models toward new languages. The work highlights the value of extensive alignment with human preferences over simply increasing model scale.
Questions people ask about ALLaM
What languages does ALLaM support?
ALLaM is specifically designed to handle both Arabic and English. It uses a mixed training approach to maintain high performance in English while acquiring strong Arabic capabilities.
Who developed the ALLaM model?
The National Center for Artificial Intelligence at the Saudi Data and AI Authority developed ALLaM. The goal is to advance Arabic Language Technology through open research and development.
How many parameters does the preview model have?
The ALLaM-7B-Instruct-preview model contains 7 billion parameters. It was trained from scratch on a large dataset of English and mixed Arabic-English tokens.
Good reading and sources
There is no English Wikipedia article on ALLaM yet, so the facts on this page come from the sources listed above: huggingface.co/ALLaM-AI/ALLaM-7B-Instruct-preview, arxiv.org/abs/2407.15390. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.