CroissantLLM
A 1.3 billion parameter bilingual French-English language model.
CroissantLLM is a 1.3 billion parameter language model trained on three trillion tokens of English and French text. Released in 2024 by a research team, it aims to provide a high-performance, open-source bilingual model that runs efficiently on consumer hardware while promoting transparency in multilingual AI development.
- Trained on 3 trillion English and French tokens.
- Features a 1:1 English-to-French data ratio.
- Introduces the FrenchBench evaluation benchmark.
- Validates 81% of FMTI transparency criteria.
- Runs efficiently on consumer-grade hardware.
What it is
CroissantLLM is a large language model with 1.3 billion parameters, designed specifically for bilingual tasks in English and French. It was developed to run efficiently on standard consumer hardware, making advanced natural language processing more accessible. The model processes text inputs and generates text outputs, serving as a base for various applications including translation and chat interfaces. Its compact size allows for local deployment without requiring extensive computational resources.
How it came about
The model was introduced in February 2024 through a research paper submitted to arXiv. The project was led by Manuel Faysse and a team of fifteen other researchers. They aimed to break away from English-centric models by creating a truly bilingual system. The team released multiple checkpoints, codebases, and fine-tuned versions, including a chat variant, to support the research community. The work emphasizes transparency and reproducibility in large language model development.
Inside the model
The architecture is a decoder-only transformer trained on three trillion tokens. A key design choice is the one-to-one ratio of English to French data during pretraining, ensuring balanced bilingual capabilities. The developers created a custom tokenizer and used bilingual fine-tuning datasets. They also released the training dataset, which includes a manually curated French split. This approach aims to improve understanding of multilinguality in language models compared to previous English-dominated efforts.
What it can do
The model performs well on bilingual tasks, particularly translation between English and French. To evaluate its French capabilities, the creators developed FrenchBench, a new benchmark covering classification and generation tasks. The base model works best with few-shot prompting strategies. It is not fine-tuned for chat by default, though a separate chat version is available. The model validates eighty-one percent of transparency criteria under the FMTI framework.
How to use it
CroissantLLM is fully open-sourced, with weights available for download. The developers provide codebases and dozens of checkpoints across different training stages. Users can access the base model for custom implementations or use the fine-tuned chat version for conversational tasks. The release includes strong translation models and aims to foster further research by providing comprehensive resources and data.
Why it matters
The project contributes to the natural language processing landscape by challenging the dominance of English-only models. By releasing high-quality French data and a dedicated benchmark, it encourages more robust multilingual research. The emphasis on transparency and open access sets a precedent for future initiatives, allowing other researchers to build upon its findings and methodologies.
Questions people ask about CroissantLLM
What languages does CroissantLLM support?
CroissantLLM is a bilingual model designed specifically for English and French. It was trained on an equal mix of both languages to ensure balanced performance in translation and generation tasks for these two specific linguistic communities.
Can I run this model on my personal computer?
Yes, the model is designed to run swiftly on consumer-grade local hardware. With only 1.3 billion parameters, it is lightweight enough to be deployed on standard machines without needing specialized industrial servers or extensive computational resources.
Is there a chat version available?
Yes, while the base model is intended for few-shot prompting, the developers recommend using the Chat version for interactive conversations. This fine-tuned variant is optimized for dialogue and is available alongside the base model on Hugging Face.
Good reading and sources
There is no English Wikipedia article on CroissantLLM yet, so the facts on this page come from the sources listed above: huggingface.co/croissantllm/CroissantLLMBase, arxiv.org/abs/2402.00786. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.