Fugaku-LLM
Japanese large language model trained on Fugaku supercomputer
A consortium of Japanese institutions released this 13-billion-parameter model in May 2024. It was trained from scratch on the Fugaku supercomputer using primarily Japanese data, offering strong performance in humanities and social sciences tasks.
- Trained on 13,824 nodes of the Fugaku supercomputer
- Contains 13 billion parameters, larger than many prior Japanese models
- Achieved high scores in Japanese humanities and social sciences tasks
- Trained from scratch on 380 billion tokens, mostly in Japanese
- Open for commercial and research use under license
What it is
Fugaku-LLM is a large language model with 13 billion parameters, developed by a joint Japanese research team. Released in May 2024, it was trained entirely on the Fugaku supercomputer rather than graphics processing units. The model focuses on enhanced Japanese language capabilities while also supporting English. It serves as a resource for both academic research and commercial business applications, aiming to secure computational independence for Japan in the global artificial intelligence landscape.
How it came about
Amidst a global shortage of graphics processors, a consortium including universities and private firms initiated a project to train models on Japan’s flagship supercomputer. Led by researchers from multiple institutions, the team ported deep learning frameworks to run efficiently on Fugaku’s central processing units. This effort resulted in the release of the model in May 2024, marking a shift from continual learning of foreign models to training from scratch on domestic data.
Inside the model
The architecture relies on Transformer layers optimized for distributed parallel learning on CPUs. Engineers accelerated dense matrix multiplication and improved communication speeds across the Tofu interconnect. Training utilized 13,824 computing nodes, processing 380 billion tokens. About sixty percent of this data was in Japanese, supplemented by English, mathematics, and code. This from-scratch approach ensures transparency in the learning process, unlike models that merely fine-tune existing foreign bases.
What it can do
The model demonstrates strong performance in Japanese language tasks, particularly in humanities and social sciences, where it achieved a high benchmark score. It handles natural dialogue features such as honorific speech effectively. With 13 billion parameters, it exceeds the size of many earlier domestic models, offering a balance between computational demand and capability. It is cited as the top-performing open model trained on original Japanese data.
How to use it
Weights and source code are publicly available on platforms like GitHub and Hugging Face. Users may employ the model for research or commercial purposes provided they adhere to the license terms. Additionally, access is provided through the Fujitsu Research Portal, allowing broader engagement from engineers and scientists who wish to build applications or improve training efficiency.
Safety and testing
Training from scratch on proprietary data allows for greater transparency regarding the model’s knowledge base. This method is considered superior for safety compared to continual learning from opaque external models. The developers emphasize that understanding the entire learning process helps mitigate risks associated with unknown data sources or hidden biases inherited from other systems.
Reception and debate
The project highlights the viability of using central processing units for large-scale AI training, addressing hardware shortages. It is viewed as a significant step for Japanese semiconductor technology and national AI infrastructure. The model is expected to support innovative applications, including scientific simulations and virtual community modeling, by leveraging domestic computational resources.
Why it matters
This work demonstrates that high-performance language models can be built without relying on graphics processing units. The optimization techniques developed for Fugaku may influence future computing infrastructure design. By providing an open, domestically trained model, the project supports the growth of local AI ecosystems and reduces dependency on foreign foundational models.
Questions people ask about Fugaku-LLM
What hardware was used to train Fugaku-LLM?
The model was trained on the Fugaku supercomputer using 13,824 nodes. Unlike many modern models that use graphics processing units, this project utilized the central processing units manufactured by Fujitsu within the Fugaku system.
Is Fugaku-LLM available for commercial use?
Yes, the model is open for both research and commercial applications. Users must comply with the specified license terms. The weights and source code are accessible via public repositories and the Fujitsu Research Portal.
How does its training data differ from other Japanese models?
Most Japanese models use continual learning on top of foreign bases. Fugaku-LLM was trained from scratch using 380 billion tokens, with about sixty percent being Japanese data, ensuring transparency and better handling of native linguistic features.
Good reading and sources
There is no English Wikipedia article on Fugaku-LLM yet, so the facts on this page come from the sources listed above: fujitsu.com/global/about/resources/news/press-releases/2024/0510-01.ht. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.