News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
PLaMo-100B

PLaMo-100B

A 100-billion parameter language model built for Japanese.

Preferred Elements released PLaMo-100B in October 2024 as a large language model trained from scratch to excel in Japanese. It uses 2 trillion tokens and specific architectural choices to maintain stability, aiming to match the performance of leading global models on Japanese tasks.

Share on WhatsApp

  • Trained from scratch on 2 trillion tokens
  • Uses QK Normalization for training stability
  • Designed specifically for Japanese proficiency
  • Base model weights are openly available
  • Competitive with GPT-4 on Japanese tasks

What it is

PLaMo-100B is a large language model created by Preferred Elements. It was introduced in October 2024 with a specific focus on high proficiency in the Japanese language. The model contains 100 billion parameters and was trained from the ground up rather than being adapted from an existing English-centric base. Its primary goal is to handle Japanese-specific linguistic tasks effectively while remaining competitive with other advanced models in the field.

How it came about

The development team at Preferred Elements announced the model in October 2024 via a research paper. The project involved a group of researchers including Kenshin Abe and Daisuke Okanohara. They designed the system to address gaps in Japanese language processing by training it from scratch. The initial release included the base model weights, allowing the community to access the core technology immediately upon announcement.

Inside the model

The architecture employs a decoder-only transformer design. To ensure stability during the massive training process, the developers implemented QK Normalization and Z-Loss techniques. The model was trained on a dataset comprising 2 trillion tokens. After the initial pre-training phase, the team applied Supervised Fine-Tuning to improve instruction following. They also used Direct Preference Optimization to align the model outputs with human preferences, refining its overall performance and safety.

What it can do

Benchmark evaluations indicate that PLaMo-100B performs strongly in tasks specifically related to the Japanese language. The developers claim its results are competitive with frontier models such as GPT-4 in these areas. While it is designed for Japanese proficiency, it functions as a general-purpose language model. The performance metrics suggest it can handle complex linguistic nuances that are often challenging for models primarily trained on English data.

How to use it

The base model weights are available to the public. Preferred Elements has made them accessible through online repositories, allowing researchers and developers to download and use the model. This open approach facilitates further study and application of the technology. Users can integrate the model into their own systems or conduct additional fine-tuning for specific use cases without needing to train a large model from scratch.

Questions people ask about PLaMo-100B

Who developed PLaMo-100B?

Preferred Elements, a research organization, developed the model. A team of authors including Kenshin Abe and Daisuke Okanohara contributed to its creation and the associated research paper.

What makes this model different from others?

It was trained from scratch with a specific focus on Japanese language proficiency. It uses architectural techniques like QK Normalization and Z-Loss to ensure stability during training on 2 trillion tokens.

Is the model available for public use?

Yes, the base model weights are available for download. This allows researchers and developers to access the core model and build upon it for various applications or further fine-tuning.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on PLaMo-100B yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2410.07563. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.