RakutenAI-7B
Japanese-focused open language model based on Mistral
Rakuten Group released RakutenAI-7B in March 2024 as a suite of large language models optimised for Japanese. Built on the Mistral-7B architecture with an expanded vocabulary, it aims to lead open-source performance in Japanese benchmarks while maintaining strong English capabilities.
- Based on Mistral-7B-v0.1 architecture
- Vocabulary expanded to 48k tokens for Japanese
- Top scores on Japanese LM-Harness benchmarks
- Released under Apache License 2.0
- Includes base, instruct, and chat variants
What it is
RakutenAI-7B is a family of large language models developed by Rakuten Group. It includes a foundation model, an instruction-tuned variant, and a chat-tuned version. The system was designed to handle Japanese text more efficiently than previous open models by expanding the token vocabulary. Released in March 2024, it targets developers and researchers needing high-performance Japanese language processing under an open licence.
How it came about
The model was announced in March 2024 via a technical report on arXiv. Rakuten Group positioned it as a systematic effort to bring advanced technologies to Japanese large language models. The release included the base model alongside instruct and chat variants, all made available simultaneously to support different use cases from raw generation to conversational agents.
Inside the model
The architecture relies on the Mistral-7B-v0.1 pre-trained checkpoint. Rakuten engineers retrofitted this base by extending its vocabulary from 32,000 to 48,000 tokens. This change improves the character-per-token rate for Japanese script. The model retains the decoder-only transformer design of its predecessor. Training focused on balancing Japanese understanding with competitive English performance, using standard causal language modelling objectives.
What it can do
On Japanese LM-Harness benchmarks, the model achieved the highest average score among comparable open 7B parameter models. It scored 69.80 on the 7-Avg metric, outperforming rivals like Nekomata and Swallow. In English tests, it also led with an average of 60.50, surpassing Japanese StableLM and Elyza. Independent evaluations in March 2024 confirmed its top ranking for chat and instruct versions among similar-sized open models.
How to use it
All three variants are available with open weights under the Apache License, Version 2.0. Users can download the models from Hugging Face. The permissive licence allows for commercial and research use without strict restrictions. Code examples for loading the model via the Transformers library are provided in the documentation.
Safety and testing
The developers note that the models can generate human-like text but may produce biased or inaccurate outputs. They advise users to exercise caution and judgement. No specific red-teaming results or safety mitigation techniques beyond standard warnings are detailed in the provided materials.
Reception and debate
The model was received as a significant step for open Japanese language AI. Benchmarks showed it leading existing open options in both Japanese and English tasks. The independent Nejumi LLM Leaderboard Neo confirmed its high standing in March 2024, citing weighted averages from llm-jp-eval and Japanese MT-bench.
Why it matters
RakutenAI-7B demonstrated that retrofitting existing architectures like Mistral could yield state-leading results for specific languages. Its open licence encourages further development in the Japanese AI community. By improving token efficiency for Japanese, it sets a new baseline for future open models targeting the language.
Questions people ask about RakutenAI-7B
What is the base architecture of RakutenAI-7B?
It is built on the Mistral-7B-v0.1 pre-trained checkpoint. The developers retrofitted this base model by extending its vocabulary to better handle Japanese characters.
Is RakutenAI-7B free to use?
Yes, the model weights are released under the Apache License, Version 2.0. This permissive licence allows for both commercial and research applications without heavy restrictions.
How does it perform in English?
It maintains competitive performance in English. On English LM-Harness metrics, it achieved an average score of 60.50, which was higher than other comparable Japanese-focused models.
Good reading and sources
There is no English Wikipedia article on RakutenAI-7B yet, so the facts on this page come from the sources listed above: huggingface.co/Rakuten/RakutenAI-7B, arxiv.org/abs/2403.15484. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.