News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
LLaDA

LLaDA

An 8-billion parameter diffusion language model

LLaDA is an 8-billion parameter language model built on diffusion architecture rather than autoregression. Released in 2025 by researchers including Shen Nie and Chongxuan Li, it challenges the assumption that large language capabilities require next-token prediction. The model matches LLaMA3 8B in general tasks and instruction following.

Share on WhatsApp

  • Uses diffusion instead of autoregressive next-token prediction
  • 8 billion parameters trained entirely from scratch
  • Matches LLaMA3 8B in general task benchmarks
  • Surpasses GPT-4o in reversal poem completion
  • Open weights available on Hugging Face

What it is

LLaDA is a large language model with eight billion parameters that uses a diffusion process instead of the standard autoregressive method. It was introduced in early 2025 by a team of researchers associated with GSAI-ML. The model aims to prove that high-level language understanding and generation do not strictly depend on predicting the next word in a sequence. By employing a masking and unmasking strategy, it generates text through iterative refinement. The 8B instruct version is designed to follow user commands and engage in dialogue, showing performance comparable to established models like LLaMA3 8B.

How it came about

The project was first described in a paper submitted to arXiv in February 2025, titled Large Language Diffusion Models. The work was led by Shen Nie, Chongxuan Li, and colleagues including Fengqi Zhu and Ji-Rong Wen. They developed the model from scratch to test the scalability of diffusion approaches in natural language processing. An updated version of the code was released in October 2025 to support attention masks, improving its technical flexibility. The release included both the base pre-trained model and a supervised fine-tuned variant for instruction following.

Inside the model

Unlike traditional models that predict tokens one by one, LLaDA uses a forward masking process where parts of the data are hidden. A Transformer network then predicts these masked tokens during a reverse generation process. This approach optimizes a likelihood lower bound for probabilistic inference. The model was trained using pre-training and supervised fine-tuning paradigms. It does not rely on autoregressive baselines but constructs its own for comparison. The architecture allows it to address specific limitations found in sequential models, such as the reversal curse, by considering context more holistically during generation.

What it can do

The model demonstrates strong scalability across general tasks, mathematics, and coding benchmarks. In evaluations, the 8B parameter version performs comparably to LLaMA3 8B in in-context learning scenarios. After supervised fine-tuning, it shows robust instruction-following capabilities in multi-turn dialogues. Notably, the researchers claim it surpasses GPT-4o in reversal poem completion tasks, indicating a better grasp of bidirectional relationships in text. These results suggest that diffusion models can match the core competencies of leading autoregressive systems.

How to use it

The model weights and code are available openly through Hugging Face and GitHub. The developers provide an 8B instruct variant for direct use in chat and instruction-based applications. A project page offers demonstrations of its capabilities. The codebase was updated in late 2025 to include support for attention masks, allowing for more flexible integration into existing pipelines. Users can access the implementation to run the model locally or adapt it for further research.

Why it matters

LLaDA challenges the dominant view that autoregressive models are the only viable path for large-scale language intelligence. By showing competitive performance with a diffusion-based approach, it opens new avenues for architectural diversity in natural language processing. The work suggests that probabilistic inference via masking can be scaled effectively, potentially offering alternatives to next-token prediction for future model development.

Questions people ask about LLaDA

How does LLaDA generate text differently from other models?

Instead of predicting one word at a time from left to right, LLaDA uses a diffusion process. It starts with masked data and iteratively predicts the missing tokens using a Transformer, allowing it to consider the whole context simultaneously rather than sequentially.

Is LLaDA as powerful as LLaMA3?

The 8B parameter version of LLaDA is reported to be competitive with LLaMA3 8B in in-context learning and general benchmarks. After fine-tuning, it also shows strong instruction-following abilities comparable to established autoregressive models.

Can I download and run LLaDA myself?

Yes, the model weights and code are openly available. You can find the 8B-Instruct variant on Hugging Face and the implementation code on GitHub, allowing for local deployment and further experimentation.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on LLaDA yet, so the facts on this page come from the sources listed above: huggingface.co/GSAI-ML/LLaDA-8B-Instruct, arxiv.org/abs/2502.09992. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.