Nemotron-4 340B
NVIDIA's 340B open-access model family for chat and synthetic data generation
Nemotron-4 340B is a family of large language models released by NVIDIA in June 2024, comprising base, instruct and reward variants. The company says the models are competitive with other open access models and are aimed mainly at producing synthetic training data for smaller language models, with over 98% of the alignment data generated synthetically.
- 340 billion parameter family: base, instruct and reward models
- Pretrained on 9 trillion tokens, 50+ natural and 40+ coding languages
- Over 98% of alignment data was synthetically generated
- Runs on a single DGX H100 with 8 GPUs in FP8 precision
- Released under the NVIDIA Open Model License in June 2024
What it is
Nemotron-4 340B is a family of large language models from NVIDIA, published in June 2024. It has three members: a base model, an instruct version tuned for chat, and a reward model. The instruct variant is a fine-tune of the base model aimed at English single and multi-turn conversation, with a 4,096 token context window. NVIDIA sized the models to run on one DGX H100 with 8 GPUs in FP8 precision. The stated purpose is helping researchers and companies generate synthetic data for training and customising their own language models.
How it came about
The technical report was submitted to arXiv on 17 June 2024, with a revision in August 2024, and lists more than 80 NVIDIA authors. Training of the instruct model ran from December 2023 to May 2024, and the pretraining data has a June 2023 cutoff. Alongside the models, NVIDIA open-sourced the synthetic data generation pipeline used during alignment to support further research.
Inside the model
The models use a standard decoder-only Transformer design, trained at a sequence length of 4,096 tokens with grouped-query attention and rotary position embeddings. Pretraining covered 9 trillion tokens drawn from English text, more than 50 natural languages and over 40 coding languages. Alignment then added Direct Preference Optimization and Reward-aware Preference Optimization, an in-house NVIDIA method. Only around 20,000 human-annotated examples were used; the pipeline synthesised more than 98% of the data for supervised and preference fine-tuning.
What it can do
NVIDIA reports that the models perform competitively against other open access models across a range of evaluation benchmarks. The instruct model is described as aligned to human chat preferences, with gains in mathematical reasoning, coding and instruction following, and as able to produce high quality synthetic data. On MT-Bench, judged by GPT-4-0125-Preview, it scored 8.22 overall, with 8.70 for writing, 8.40 for math and 6.70 for coding. The model card also lists IFEval, MMLU, GSM8K, HumanEval, MBPP, Arena Hard, AlpacaEval 2.0 LC and TFEval among the evaluations run.
How to use it
The weights are released under the NVIDIA Open Model License, which the company says permits commercial use, distribution and modification, and claims no ownership of outputs. NVIDIA also offers the instruct model through build.nvidia.com. Running it locally needs 16 A100 80GB GPUs across two nodes, and the documented deployment path uses the NeMo Framework, NeMo-Aligner and Slurm. Further customisation is supported through parameter-efficient fine-tuning such as LoRA and adapters.
Safety and testing
The instruct model went through adversarial testing by three routes: Garak, an automated scanner probing for weaknesses such as prompt injection and data leakage; AEGIS, a content safety dataset and classifier covering 13 risk categories; and human red teaming, where people interacted with the model and judged its replies. The model card notes that the training data included toxic and unsafe content, so the model can produce such outputs.
Why it matters
NVIDIA positions the family as a source of synthetic training data for smaller language models, and released its own data generation pipeline openly to help that work. The company says more than 98% of the data used to align the models was synthesised, which it presents as evidence that models of this kind can generate useful training data.
Questions people ask about Nemotron-4 340B
What is Nemotron-4 340B for?
It is mainly meant to generate synthetic training data so that researchers and companies can build and customise their own language models. The instruct version also handles English chat, and NVIDIA says it is tuned for instruction following, coding and maths.
Who made it and when did it come out?
NVIDIA released the family in June 2024, with the technical report posted on 17 June 2024. Training of the instruct model took place between December 2023 and May 2024, and the pretraining data stops at June 2023.
Can I use it commercially?
Yes, according to NVIDIA. The weights come under the NVIDIA Open Model License, which the company says allows commercial use, distribution and modification of the models and their outputs, and takes no ownership of generated outputs.
What hardware do I need to run it?
The model card lists 16 A100 80GB GPUs across two nodes as the requirement. NVIDIA says the models were sized to fit a single DGX H100 with 8 GPUs when served in FP8 precision.
How was it aligned to human preferences?
Alignment used Direct Preference Optimization and Reward-aware Preference Optimization, an in-house NVIDIA technique. Roughly 20,000 human-annotated examples were used, while the pipeline generated more than 98% of the data for supervised and preference fine-tuning.
Good reading and sources
There is no English Wikipedia article on Nemotron-4 340B yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2406.11704, huggingface.co/nvidia/Nemotron-4-340B-Instruct. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.