News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
LLM-jp-3

LLM-jp-3

Open Japanese language model family by NII

LLM-jp-3 is a series of large language models developed by the National Institute of Informatics in Japan. Released in 2024, it includes sizes from 1.8 billion to 172 billion parameters. The project aims to provide fully open, strong Japanese LLMs through cross-organizational collaboration.

Share on WhatsApp

  • Largest model has 172 billion parameters
  • Trained on Japanese, English, and code data
  • Context length limited to 4096 tokens
  • Fully open weights available on Hugging Face
  • Supported by GENIAC and 1500+ participants

What it is

LLM-jp-3 is a family of transformer-based language models created by the Research and Development Center for Large Language Models at the National Institute of Informatics. The series ranges from small 1.8 billion parameter versions to a massive 172 billion parameter model. It was developed as part of a broader cross-organizational effort to build open-source Japanese AI. The largest model supports a context window of 4096 tokens and uses a decoder-only architecture.

How it came about

The project involves over 1,500 participants from academia and industry working together. Development was partially supported by GENIAC, a Japanese government initiative. The team released multiple checkpoints in 2024, including beta versions and instruct-tuned variants. The work builds on previous efforts to create fully open Japanese large language models.

Inside the model

The models use a standard transformer decoder design with layer counts scaling from 24 to 96. Training data includes billions of tokens from Japanese sources like Wikipedia and Common Crawl, as well as English, code, Chinese, and Korean texts. Post-training involved supervised fine-tuning on instruction datasets and alignment via Direct Preference Optimization. The tokenizer is based on Unigram byte-fallback with a custom vocabulary.

What it can do

Evaluation on the llm-jp-eval benchmark shows the 172b instruct model achieving an average score of 0.6130. It performs strongly in reading comprehension and natural language inference. On the Japanese MT Bench, the same model scored 7.57 on average, with high marks in writing and humanities. Code generation and math reasoning scores were lower compared to other categories.

How to use it

All model weights are available openly on Hugging Face under permissive terms. Users can download checkpoints for sizes including 1.8b, 3.7b, 13b, and 172b. Both base and instruct-tuned versions are provided. The models are compatible with standard libraries like Hugging Face Transformers for easy integration.

Safety and testing

The developers note that the models are in early stages of research. They have not been fully tuned to ensure outputs align with human values. Safety-focused instruction datasets were used during fine-tuning, but risks remain. Users are advised to consider these limitations when deploying the models.

Why it matters

This project represents a significant collaborative effort in Japanese AI research. By providing fully open weights, it allows other researchers and developers to build upon the work. It contributes to the availability of high-quality language models specifically optimized for Japanese and multilingual contexts.

Questions people ask about LLM-jp-3

Who developed LLM-jp-3?

The Research and Development Center for Large Language Models at the National Institute of Informatics in Japan developed it. Over 1,500 participants from various organizations contributed to the project.

What languages does it support?

It is primarily designed for Japanese but was trained on a mix of Japanese, English, code, Chinese, and Korean data. This multilingual training helps it handle diverse inputs.

Is the model free to use?

Yes, the model weights are fully open and available for download. You can access them via Hugging Face and use them with standard transformer libraries.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on LLM-jp-3 yet, so the facts on this page come from the sources listed above: huggingface.co/llm-jp/llm-jp-3-172b-instruct3, arxiv.org/abs/2407.03963. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.