Meltemi
Open large language model for Greek
Meltemi is an open large language model designed specifically for the Greek language. Developed by the Institute for Language and Speech Processing, it adapts the Mistral architecture with a specialised Greek vocabulary and training data.
- First open large language model for Greek
- Trained on 40 billion Greek tokens
- Uses extended Mistral tokenizer for efficiency
- Released under Apache 2.0 licence
- Includes medical question answering benchmark
What it is
Meltemi is a large language model built to handle the Greek language. It contains seven billion parameters and operates as a decoder-only transformer. The Institute for Language and Speech Processing created the system to provide an open resource for Greek speakers. It processes text inputs and generates text outputs, focusing on linguistic accuracy and cultural context specific to Greece. The model serves as a base for both general understanding and instruction-following tasks.
How it came about
The development team adapted the existing Mistral architecture for this project. They performed continuous pretraining on a corpus containing forty billion Greek tokens. This process ensured the model captured up-to-date information through September 2023. The researchers then curated a Greek instruction dataset to create a chat-oriented version. Both the base and instruct variants were released in July 2024 under an open licence.
Inside the model
The architects extended the Mistral tokenizer with specific Greek tokens. This change reduced the cost and speed of inference significantly for Greek text. Training used the Odds Ratio Preference Optimization algorithm on nearly ninety thousand preference pairs. Most of these data points were high-quality translations from existing datasets. The alignment procedure relied on the Transformer Reinforcement Learning library to refine responses.
What it can do
Evaluation involved six test sets, including translated versions of ARC and MMLU. A novel benchmark tested medical question answering using Greek exam data. The instruct version showed a seven point eight percent average improvement over its predecessor. It scored forty-eight percent on medical queries and seventy-five point five percent on general question answering. The developers note it may still produce misleading or toxic content despite alignment efforts.
How to use it
Users can download the model weights from Hugging Face. The software is distributed under the Apache 2.0 licence, allowing broad usage. The code supports standard chat templates for easy integration into applications. Developers must ensure the beginning-of-sequence token is included during processing. Cloud computing resources were provided through the Greek academic network.
Safety and testing
The team prioritised the removal of toxic content during the instruction-tuning phase. They aligned the model with human preferences using a curated dataset. Despite these measures, the documentation warns that the system might generate harmful or misleading material. Users are advised to exercise caution when deploying the model in sensitive environments.
Reception and debate
The creators describe it as the first open large language model for Greek. Academic peers can cite the work through the associated arXiv paper. The community has since been directed toward a newer model named Krikri, which reportedly outperforms Meltemi on Greek benchmarks. This suggests the field is moving quickly beyond this initial release.
Why it matters
This project established a baseline for open-source Greek language processing. It demonstrated how existing architectures like Mistral could be effectively adapted for lower-resource languages. The release of the evaluation suite provides a standard for future models to measure their performance against. It encourages further research into Greek-specific natural language understanding tasks.
Questions people ask about Meltemi
What language does Meltemi support?
Meltemi is designed specifically for the Greek language. It uses a custom vocabulary extension to handle Greek text more efficiently than standard multilingual models.
Is Meltemi free to use?
Yes, the model weights are available under the Apache 2.0 licence. This permits open use, modification, and distribution for both commercial and non-commercial purposes.
How was Meltemi trained?
It was built by adapting the Mistral architecture. The team performed continuous pretraining on a forty billion token Greek corpus and fine-tuned it using preference data.
Who developed Meltemi?
The Institute for Language and Speech Processing (ILSP) developed the model. The team utilized cloud computing services provided via the Greek academic network.
Good reading and sources
There is no English Wikipedia article on Meltemi yet, so the facts on this page come from the sources listed above: huggingface.co/ilsp/Meltemi-7B-Instruct-v1.5, arxiv.org/abs/2407.20743. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.