News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
imgMolmo

Molmo

Open vision-language model family by Allen Institute for AI

The Allen Institute for AI released Molmo, a set of open vision-language models. Trained on the PixMo dataset without external proprietary data, it matches or beats closed rivals in tests.

Share on WhatsApp

  • Trained on PixMo, a dataset of 1 million curated pairs
  • 72B model outperforms Claude 3.5 Sonnet and Gemini 1.5 Pro
  • Uses Qwen2-7B and OpenAI CLIP as core components
  • Released under Apache 2.0 licence for open use
  • Collected data without using external proprietary VLMs

What it is

Molmo is a collection of vision-language systems created by the Allen Institute for AI. These models process both images and text to answer questions or describe visual content. The project aims to provide high-performance alternatives to closed commercial systems. It includes various sizes, with the largest reaching 72 billion parameters. The team released the weights and training data to support open research.

How it came about

Researchers announced the project in September 2024 via an academic paper. They highlighted a gap in the field, noting that top open models often relied on synthetic data from private competitors. To address this, they built new datasets from scratch. The initial release included a 7B parameter variant as a preview. Further artifacts were promised for later distribution to ensure reproducibility.

Inside the model

The architecture combines a language decoder with a visual encoder. The 7B version uses Qwen2 as its text base and OpenAI CLIP for image processing. Training relied on PixMo, a curated set of one million image-text pairs. This dataset includes detailed captions and pointing tasks, collected without help from other large models. The pipeline emphasizes careful modeling choices over sheer scale.

What it can do

Performance rivals leading proprietary tools. In human evaluations and standard tests, the 72B model surpassed Claude 3.5 Sonnet and Gemini 1.5 Pro. It ranked second only to GPT-4o. Smaller versions also showed strong results for their size class. The system handles free-form questions and detailed image analysis effectively.

How to use it

Weights are available under the Apache 2.0 licence. This permits research and educational use. The code and datasets are shared publicly. Users can run the model locally using libraries like transformers or vLLM. A web demo is hosted by the institute for quick testing.

Safety and testing

The institute provides responsible use guidelines. These documents outline intended applications and potential risks. The open nature allows the community to audit the model behavior. No specific safety incidents or red-teaming results are detailed in the provided texts.

Reception and debate

The release was noted for breaking reliance on proprietary distillation. By providing clean data and weights, it offered a new baseline for open development. Reviewers highlighted its competitive scores against closed giants. The approach was seen as a significant step for transparent AI research.

Why it matters

Molmo demonstrates that open models can reach state-of-the-art status without secret data. It provides a foundation for others to build upon. The PixMo dataset serves as a resource for training future vision systems. This may reduce the dominance of closed providers in multimodal tasks.

Questions people ask about Molmo

What makes Molmo different from other open models?

It avoids using synthetic data generated by private competitors. Instead, it relies on PixMo, a newly collected dataset of image-text pairs. This ensures the training knowledge is original and not distilled from closed sources.

How does its performance compare to GPT-4o?

The largest 72B version ranks second to GPT-4o in human evaluations and benchmarks. It surpasses other major proprietary models like Claude 3.5 Sonnet and Gemini 1.5 Pro, showing competitive capability despite being open.

Can I use Molmo for commercial products?

The model is licensed under Apache 2.0. This permissive licence generally allows commercial use, though the institute states it is intended for research and education. Users should check the specific responsible use guidelines for details.

What technical base does the 7B model use?

The 7B variant builds on the Qwen2-7B language model. For visual understanding, it integrates the OpenAI CLIP vision encoder. This hybrid design allows it to process images and text together effectively.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on Molmo yet, so the facts on this page come from the sources listed above: huggingface.co/allenai/Molmo-7B-D-0924, arxiv.org/abs/2409.17146. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.