CyberAgentLM3
A 22-billion parameter Japanese and English chat model by CyberAgent.
CyberAgentLM3 is a decoder-only language model developed by CyberAgent, Inc. It was pre-trained on 2.0 trillion tokens from scratch. The Chat variant is fine-tuned for dialogue and supports Japanese and English.
- Pre-trained on 2.0 trillion tokens from scratch.
- Supports both Japanese and English languages.
- Uses ChatML format for prompt structure.
- Available as open weights on Hugging Face.
What it is
CyberAgentLM3 is a large language model built by CyberAgent, Inc. It functions as a decoder-only transformer. The base model underwent pre-training on a dataset of 2.0 trillion tokens. A specific variant, CALM3-22B-Chat, exists for conversational tasks. This version receives fine-tuning to handle dialogue interactions effectively. The system processes both Japanese and English text inputs.
Inside the model
The architecture relies on a standard decoder-only transformer design. Training involved processing 2.0 trillion tokens from the ground up. For conversational applications, the model undergoes further fine-tuning. This process creates the CALM3-22B-Chat variant. The interface uses the ChatML format for structuring prompts. This format helps define roles such as system and user within the input sequence.
What it can do
The model supports generation in Japanese and English. It is optimized for dialogue use cases through fine-tuning. Users can interact with it via standard transformer libraries. The system accepts role-based prompts to guide its responses. It generates text completions based on the provided context window.
How to use it
Weights are available for download via Hugging Face. Developers can load the model using the transformers library. The code examples show usage with AutoModelForCausalLM. This allows local deployment on compatible hardware. The model is accessible for research and application development without an API gate.
Questions people ask about CyberAgentLM3
What languages does CyberAgentLM3 support?
The model is designed to process and generate text in both Japanese and English. It handles these two languages specifically according to its configuration details.
How large is the training dataset?
CyberAgentLM3 was pre-trained on a corpus containing 2.0 trillion tokens. This data was used to train the model from scratch before any fine-tuning occurred.
Is this model suitable for chat applications?
Yes, the CALM3-22B-Chat variant is fine-tuned specifically for dialogue use cases. It uses the ChatML prompt format to manage conversation structures effectively.
Good reading and sources
There is no English Wikipedia article on CyberAgentLM3 yet, so the facts on this page come from the sources listed above: huggingface.co/cyberagent/calm3-22b-chat. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.