PhoGPT
Open-source generative model for Vietnamese language tasks.
VinAI Research released PhoGPT, a 3.7 billion parameter model trained specifically for Vietnamese. It includes a base version and a chat variant fine-tuned on instructional data, offering open weights for the community.
- Contains exactly 3.7 billion parameters.
- Trained on 102 billion Vietnamese tokens.
- Supports a context length of 8192 tokens.
- Chat variant fine-tuned on 360K examples.
- Open-sourced by VinAI Research.
What it is
PhoGPT is a generative pre-trained model series designed specifically for the Vietnamese language. Developed by VinAI Research, it aims to provide high-quality language processing capabilities for this specific linguistic context. The series includes a base model called PhoGPT-4B and a conversational variant named PhoGPT-4B-Chat. The base model contains exactly 3.7 billion parameters. It was built from scratch using a large corpus of Vietnamese text. The project is open-sourced to support further research and application development in the region.
How it came about
The technical report for PhoGPT was first submitted to arXiv in November 2023. The authors include Dat Quoc Nguyen, Linh The Nguyen, Chi Tran, Dung Ngoc Nguyen, Dinh Phung, and Hung Bui. The paper underwent revisions in February and March 2024. VinAI Research made the models available through Hugging Face and their GitHub repository. The release marked a significant step in providing dedicated large language models for Vietnamese speakers.
Inside the model
The base model, PhoGPT-4B, uses a decoder-only architecture. It was pre-trained on a dataset comprising 102 billion tokens of Vietnamese text. The model employs a vocabulary of 20,480 token types and supports a context window of 8,192 tokens. The chat variant, PhoGPT-4B-Chat, was created by fine-tuning the base model. This process used 70,000 instructional prompts with responses and an additional 290,000 conversations. This alignment step helps the model follow instructions and engage in dialogue more effectively.
What it can do
The developers state that PhoGPT demonstrates superior performance compared to previous open-source models for Vietnamese tasks. As a monolingual model, it is optimized for understanding and generating Vietnamese text. The chat variant is capable of handling instructional prompts and multi-turn conversations. The model is intended for use in applications requiring high-quality Vietnamese language generation and comprehension.
How to use it
VinAI Research has open-sourced both the base and chat variants of PhoGPT. The weights are available for download via Hugging Face. Users can access the code and further details through the project's GitHub homepage. The team requests that users cite their technical report when using the model for published results or incorporating it into other software.
Questions people ask about PhoGPT
What languages does PhoGPT support?
PhoGPT is a monolingual model designed specifically for Vietnamese. It was pre-trained from scratch on a large corpus of Vietnamese text to optimize performance for this language.
How many parameters does the model have?
The base model, PhoGPT-4B, has exactly 3.7 billion parameters. Despite the name suggesting four billion, the precise count provided by the developers is 3.7 billion.
Is PhoGPT available for commercial use?
The model is open-sourced with open weights available on Hugging Face. Users should check the specific licence terms on the repository, but the developers encourage citation when used in published work.
Good reading and sources
There is no English Wikipedia article on PhoGPT yet, so the facts on this page come from the sources listed above: huggingface.co/vinai/PhoGPT-4B-Chat, arxiv.org/abs/2311.02945. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.