News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
Fugaku-LLM

Fugaku-LLM

Japanese large language model trained on Fugaku supercomputer

A consortium of Japanese institutions released this 13-billion-parameter model in May 2024. It was trained from scratch on the Fugaku supercomputer using primarily Japanese data, offering strong performance in humanities and social sciences tasks.

Share on WhatsApp

  • Trained on 13,824 nodes of the Fugaku supercomputer
  • Contains 13 billion parameters, larger than many prior Japanese models
  • Achieved high scores in Japanese humanities and social sciences tasks
  • Trained from scratch on 380 billion tokens, mostly in Japanese
  • Open for commercial and research use under license

What it is

Fugaku-LLM is a large language model with 13 billion parameters, developed by a joint Japanese research team. Released in May 2024, it was trained entirely on the Fugaku supercomputer rather than graphics processing units. The model focuses on enhanced Japanese language capabilities while also supporting English. It serves as a resource for both academic research and commercial business applications, aiming to secure computational independence for Japan in the global artificial intelligence landscape.

How it came about

Amidst a global shortage of graphics processors, a consortium including universities and private firms initiated a project to train models on Japan’s flagship supercomputer. Led by researchers from multiple institutions, the team ported deep learning frameworks to run efficiently on Fugaku’s central processing units. This effort resulted in the release of the model in May 2024, marking a shift from continual learning of foreign models to training from scratch on domestic data.

Inside the model

The architecture relies on Transformer layers optimized for distributed parallel learning on CPUs. Engineers accelerated dense matrix multiplication and improved communication speeds across the Tofu interconnect. Training utilized 13,824 computing nodes, processing 380 billion tokens. About sixty percent of this data was in Japanese, supplemented by English, mathematics, and code. This from-scratch approach ensures transparency in the learning process, unlike models that merely fine-tune existing foreign bases.

What it can do

The model demonstrates strong performance in Japanese language tasks, particularly in humanities and social sciences, where it achieved a high benchmark score. It handles natural dialogue features such as honorific speech effectively. With 13 billion parameters, it exceeds the size of many earlier domestic models, offering a balance between computational demand and capability. It is cited as the top-performing open model trained on original Japanese data.

How to use it

Weights and source code are publicly available on platforms like GitHub and Hugging Face. Users may employ the model for research or commercial purposes provided they adhere to the license terms. Additionally, access is provided through the Fujitsu Research Portal, allowing broader engagement from engineers and scientists who wish to build applications or improve training efficiency.

Safety and testing

Training from scratch on proprietary data allows for greater transparency regarding the model’s knowledge base. This method is considered superior for safety compared to continual learning from opaque external models. The developers emphasize that understanding the entire learning process helps mitigate risks associated with unknown data sources or hidden biases inherited from other systems.

Reception and debate

The project highlights the viability of using central processing units for large-scale AI training, addressing hardware shortages. It is viewed as a significant step for Japanese semiconductor technology and national AI infrastructure. The model is expected to support innovative applications, including scientific simulations and virtual community modeling, by leveraging domestic computational resources.

Why it matters

This work demonstrates that high-performance language models can be built without relying on graphics processing units. The optimization techniques developed for Fugaku may influence future computing infrastructure design. By providing an open, domestically trained model, the project supports the growth of local AI ecosystems and reduces dependency on foreign foundational models.

Questions people ask about Fugaku-LLM

What hardware was used to train Fugaku-LLM?

The model was trained on the Fugaku supercomputer using 13,824 nodes. Unlike many modern models that use graphics processing units, this project utilized the central processing units manufactured by Fujitsu within the Fugaku system.

Is Fugaku-LLM available for commercial use?

Yes, the model is open for both research and commercial applications. Users must comply with the specified license terms. The weights and source code are accessible via public repositories and the Fujitsu Research Portal.

How does its training data differ from other Japanese models?

Most Japanese models use continual learning on top of foreign bases. Fugaku-LLM was trained from scratch using 380 billion tokens, with about sixty percent being Japanese data, ensuring transparency and better handling of native linguistic features.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Fugaku-LLM yet, so the facts on this page come from the sources listed above: fujitsu.com/global/about/resources/news/press-releases/2024/0510-01.ht. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.