LLM-jp-3
Open Japanese language model family by NII
LLM-jp-3 is a series of large language models developed by the National Institute of Informatics in Japan. Released in 2024, it includes sizes from 1.8 billion to 172 billion parameters. The project aims to provide fully open, strong Japanese LLMs through cross-organizational collaboration.
- Largest model has 172 billion parameters
- Trained on Japanese, English, and code data
- Context length limited to 4096 tokens
- Fully open weights available on Hugging Face
- Supported by GENIAC and 1500+ participants
What it is
LLM-jp-3 is a family of transformer-based language models created by the Research and Development Center for Large Language Models at the National Institute of Informatics. The series ranges from small 1.8 billion parameter versions to a massive 172 billion parameter model. It was developed as part of a broader cross-organizational effort to build open-source Japanese AI. The largest model supports a context window of 4096 tokens and uses a decoder-only architecture.
How it came about
The project involves over 1,500 participants from academia and industry working together. Development was partially supported by GENIAC, a Japanese government initiative. The team released multiple checkpoints in 2024, including beta versions and instruct-tuned variants. The work builds on previous efforts to create fully open Japanese large language models.
Inside the model
The models use a standard transformer decoder design with layer counts scaling from 24 to 96. Training data includes billions of tokens from Japanese sources like Wikipedia and Common Crawl, as well as English, code, Chinese, and Korean texts. Post-training involved supervised fine-tuning on instruction datasets and alignment via Direct Preference Optimization. The tokenizer is based on Unigram byte-fallback with a custom vocabulary.
What it can do
Evaluation on the llm-jp-eval benchmark shows the 172b instruct model achieving an average score of 0.6130. It performs strongly in reading comprehension and natural language inference. On the Japanese MT Bench, the same model scored 7.57 on average, with high marks in writing and humanities. Code generation and math reasoning scores were lower compared to other categories.
How to use it
All model weights are available openly on Hugging Face under permissive terms. Users can download checkpoints for sizes including 1.8b, 3.7b, 13b, and 172b. Both base and instruct-tuned versions are provided. The models are compatible with standard libraries like Hugging Face Transformers for easy integration.
Safety and testing
The developers note that the models are in early stages of research. They have not been fully tuned to ensure outputs align with human values. Safety-focused instruction datasets were used during fine-tuning, but risks remain. Users are advised to consider these limitations when deploying the models.
Why it matters
This project represents a significant collaborative effort in Japanese AI research. By providing fully open weights, it allows other researchers and developers to build upon the work. It contributes to the availability of high-quality language models specifically optimized for Japanese and multilingual contexts.
Questions people ask about LLM-jp-3
Who developed LLM-jp-3?
The Research and Development Center for Large Language Models at the National Institute of Informatics in Japan developed it. Over 1,500 participants from various organizations contributed to the project.
What languages does it support?
It is primarily designed for Japanese but was trained on a mix of Japanese, English, code, Chinese, and Korean data. This multilingual training helps it handle diverse inputs.
Is the model free to use?
Yes, the model weights are fully open and available for download. You can access them via Hugging Face and use them with standard transformer libraries.
Good reading and sources
There is no English Wikipedia article on LLM-jp-3 yet, so the facts on this page come from the sources listed above: huggingface.co/llm-jp/llm-jp-3-172b-instruct3, arxiv.org/abs/2407.03963. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.