News
EveryLLM

Images in

Reads photos, charts and documents. 117 models on EveryLLM so far.

Models with image input can read photographs, screenshots, charts, diagrams, handwriting and scanned documents, and answer questions about them. An image encoder, often a vision transformer, cuts the picture into small patches and turns each into a vector that the language model treats like a token. Vision is now standard in frontier models and common in open ones.

How vision was added

Early vision-language models joined a pre-trained image encoder to a pre-trained language model. DeepMind's Flamingo (2022) did this with special cross-attention layers, and LLaVA (2023) used a simple projection trained on instruction data written by GPT-4. GPT-4 accepted images from March 2023, a feature that reached ChatGPT users in September 2023. Claude 3 (March 2024), Llama 3.2 (September 2024) and Gemma 3 (March 2025) brought vision to more users.

Uses and limits

Document understanding is the largest practical use: reading invoices, forms, tables and slides, and turning scanned pages into text. Models also describe images for blind and low-vision people, starting with a partnership between OpenAI and the Be My Eyes app in 2023. Weak spots include counting objects, reading small print, precise spatial reasoning and spotting edited photographs, and models can be misled by instructions hidden in images.

Famous images in models

Still to be written: GPT-4o.

Further reading

imgStep3 馃嚚馃嚦 ChinaStep3 Multimodal reasoning model with 321 billion parametersMultimodal 路 Open weights 路 2025 imgvidIntern-S1 馃嚚馃嚦 ChinaIntern-S1 Open-source multimodal reasoning model for scienceReasoning 路 Open weights 路 2025 imgvidfnInternVL3 馃嚚馃嚦 ChinaInternVL3 Open-source multimodal large language model seriesMultimodal 路 Open weights 路 2025 imgfnUI-TARS 馃嚚馃嚦 ChinaUI-TARS Open-source multimodal agent for GUI tasksMultimodal 路 Open weights 路 2025 imgwavvidMiMo-V2.6-Pro 馃嚚馃嚦 ChinaMiMo-V2.6-Pro Xiaomi鈥檚 1T parameter multimodal model with 1M token contextMultimodal 路 Open weights imgNVLM 1.0 馃嚭馃嚫 United StatesNVLM 1.0 Open multimodal language models from NVIDIA for vision and language tasksMultimodal 路 Open weights 路 2024 imgvidNemotron 3.5 Super VL 馃嚭馃嚫 United StatesNemotron 3.5 Super VL NVIDIA's open-weight vision-language model with 120B parameters and 1M-token contextMultimodal 路 Conditional licence 路 2026 imgvidDiffusionGemma 馃嚭馃嚫 United StatesDiffusionGemma A multimodal Gemma 4 model that writes text by denoising blocks of tokens.Multimodal 路 Open weights imgvidGLM-4.1V-Thinking 馃嚚馃嚦 ChinaGLM-4.1V-Thinking Open-source vision-language model with reasoning focusMultimodal 路 Open weights 路 2025 imgPixtral 12B 馃嚝馃嚪 FrancePixtral 12B A 12B multimodal model that reads images and text without losing text skillsMultimodal 路 Open weights 路 2024 imgDeepSeek-OCR WorldDeepSeek-OCR Optical character recognition model using visual compressionMultimodal 路 Open weights 路 2025 imgwavvidReka Core 馃嚫馃嚞 SingaporeReka Core Multimodal language model by RekaMultimodal 路 API only 路 2024 imgvidfnKimi K2.5 馃嚚馃嚦 ChinaKimi K2.5 Open-source multimodal agentic model with vision and reasoning.Multimodal 路 Open weights imgSalamandra 馃嚜馃嚫 SpainSalamandra Open multilingual language models for European languagesGeneral 路 Open weights 路 2025 imgvidfnKimi K3 馃嚚馃嚦 ChinaKimi K3 Open-weight multimodal agentic model with 2.8T parametersGeneral 路 Open weights 路 2026 imgwavvidQwen2.5-Omni 馃嚚馃嚦 ChinaQwen2.5-Omni End-to-end multimodal model for real-time speech and video chat.Multimodal 路 Open weights 路 2025 imgLLaVA 馃嚭馃嚫 United StatesLLaVA Open-source multimodal chatbot connecting vision encoders to language models.Multimodal 路 Fully open 路 2023 imgPaLM-E WorldPaLM-E An embodied multimodal language model that grounds words in robot sensor inputMultimodal 路 Research only 路 2023 imgwavGemini 2.0 Flash 馃嚭馃嚫 United StatesGemini 2.0 Flash Google DeepMind's fast multimodal workhorse model for agentic appsGeneral 路 API only 路 2024 imgvidfnStep-5-Preview WorldStep-5-Preview Agentic model with 600B parameters and 1M token contextGeneral 路 Open weights 路 2026 imgfnLeanstral 1.5 馃嚝馃嚪 FranceLeanstral 1.5 An open code agent model built for the Lean 4 proof assistantCode 路 Open weights imgAmazon Nova 馃嚭馃嚫 United StatesAmazon Nova Amazon's family of foundation models on AWS BedrockGeneral 路 API only 路 2024 imgfnMistral Large 3 馃嚝馃嚪 FranceMistral Large 3 A 675B-parameter open-weight mixture-of-experts multimodal model from Mistral AIGeneral 路 Open weights imgMM1 WorldMM1 A family of multimodal language models up to 30B parameters, dense and MoE.Multimodal 路 Research only 路 2024 imgApple Foundation Models 馃嚭馃嚫 United StatesApple Foundation Models Apple's 2025 on-device and server language models for Apple IntelligenceGeneral 路 API only 路 2025 imgGrok-2 馃嚭馃嚫 United StatesGrok-2 xAI's 2024 chat and reasoning model, with a smaller sibling, Grok-2 miniGeneral 路 Conditional licence 路 2024 imgAya Vision WorldAya Vision Open-weights multilingual multimodal vision model from Cohere For AIMultimodal 路 Open weights imgClaude 3 Opus 馃嚭馃嚫 United StatesClaude 3 Opus Anthropic's largest Claude 3 model, aimed at complex reasoning and analysisGeneral 路 API only imgClaude 3 Haiku 馃嚭馃嚫 United StatesClaude 3 Haiku Anthropic's fastest and lowest-cost model in the Claude 3 familyGeneral 路 API only 路 2024 imgwavvidGemini 1.5 Pro 馃嚭馃嚫 United StatesGemini 1.5 Pro Google DeepMind's mid-size multimodal model with up to a 1 million token context windowGeneral 路 API only 路 2024 imgwavvidGemini 1.0 馃嚭馃嚫 United StatesGemini 1.0 Google's first Gemini models: multimodal AI in Ultra, Pro and Nano sizesMultimodal 路 API only 路 2023 imgwavvidGemini 3 Pro 馃嚭馃嚫 United StatesGemini 3 Pro Google DeepMind's Gemini 3 Pro, a multimodal model with a 1 million-token context window.General 路 API only 路 2025 imgwavvidGemini 3.1 Flash-Lite 馃嚭馃嚫 United StatesGemini 3.1 Flash-Lite A fast, low-cost multimodal reasoning model for high-volume tasksMultimodal 路 API only 路 2026 imgvidfnGemma 3 馃嚭馃嚫 United StatesGemma 3 Lightweight multimodal open models from 1B to 27B parametersGeneral 路 Open weights 路 2025 imgwavvidGemma 3n 馃嚭馃嚫 United StatesGemma 3n Mobile-first open model family for on-device text, image and audio workMultimodal 路 Open weights 路 2025 imgwavvidGemma 4 馃嚭馃嚫 United StatesGemma 4 Google DeepMind's open multimodal models in five sizes, from phones to serversGeneral 路 Open weights imgPaliGemma WorldPaliGemma A 3B open vision-language model built on SigLIP and Gemma-2BMultimodal 路 Open weights 路 2024 imgMedGemma 馃嚭馃嚫 United StatesMedGemma Medical vision-language models built on Gemma 3 for images and textMultimodal 路 Conditional licence 路 2025 imgvidFlamingo 馃嚞馃嚙 United KingdomFlamingo A visual language model family that learns new image and video tasks from a few examples.Multimodal 路 Research only 路 2022 imgLlama 3.2 馃嚭馃嚫 United StatesLlama 3.2 Small text models and vision models from Meta for edge devices and image tasksMultimodal 路 Open weights 路 2024 imgChameleon WorldChameleon Early-fusion models that read and write images and text in one streamMultimodal 路 Research only 路 2024 imgwavChatGPT 馃嚭馃嚫 United StatesChatGPT Generative AI chatbot by OpenAIAssistant 路 API only 路 2022

The weekly digest

Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.