News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
imgwavvidVideoPoet

VideoPoet

A Google Research language model that turns text and images into video and audio

VideoPoet is a large language model from Google Research, announced on 19 December 2023, that generates video and audio from text prompts or still pictures. It works autoregressively and can animate a static image. The work won a Best Paper Award at ICML 2024.

Share on WhatsApp

  • Announced by Google Research on 19 December 2023
  • Generates video and sound from text prompts or still images
  • Animates a static picture on request
  • Built as an autoregressive language model
  • Best Paper Award at ICML 2024

What it is

VideoPoet is a large language model built by Google Research to produce moving pictures. Rather than only writing text, it outputs video and sound. A user can type a description and receive a clip, or hand it a still photograph and ask for that picture to be brought to life. Text, images and existing videos all work as inputs, and the system can convert between these forms. Google Research made the project public on 19 December 2023. The paper behind it later earned a Best Paper Award at ICML 2024.

How it came about

Google Research created VideoPoet during 2023 and presented it to the public on 19 December of that year. The model was offered as a demonstration of what an autoregressive language model could do with video rather than as a shipped product. Its accompanying research paper was recognised with a Best Paper Award at the ICML 2024 conference.

Inside the model

The system is built as an autoregressive language model, meaning it predicts the next piece of output step by step in sequence, the same basic approach used for text generation. Applied to video, that sequential prediction covers moving pictures and accompanying sound. The model takes in text, still images and video, and a programme layer lets any of those inputs be turned into any of the generated output formats. Detailed figures for layers, parameter count, training data and hardware are not given here.

What it can do

VideoPoet can generate clips from written descriptions and can animate a still image so that it moves. Sample outputs shown include a dog eating popcorn at the cinema and a teddy bear in a cap, sunglasses and leather jacket playing drums, both with sound. Because it accepts images and video as well as text, it can work across input and output types. No benchmark scores are reported here.

How to use it

VideoPoet was presented as a research project from Google Research. No public weights, licence terms or hosted product are described here.

Reception and debate

The research community recognised the work with a Best Paper Award at ICML 2024. No other reception is described here.

Why it matters

VideoPoet showed that a single autoregressive language model could handle video and audio generation alongside text and images, rather than needing separate systems for each output type.

Questions people ask about VideoPoet

What does VideoPoet do?

It is a large language model that produces video and audio. You can give it a written description and get a clip back, or give it a still photograph and ask for that image to be animated. It also accepts video as input.

Who made it and when?

Google Research developed it during 2023 and announced it publicly on 19 December 2023. The project is a research effort rather than a commercial product.

How does it work?

It uses an autoregressive language model, which predicts output one piece at a time in sequence. A programme layer allows any accepted input, text, image or video, to be turned into any of the generated output formats.

Has it won any awards?

Yes. The work behind VideoPoet received a Best Paper Award at ICML 2024, the International Conference on Machine Learning.

Can I download or use it?

No public release of weights, licence terms or a hosted service is described here. It was presented as research from Google Research.

Good reading and sources

Where this page comes from

The facts on this page come from the Wikipedia article "VideoPoet", written by its contributors and used under the CC BY-SA 4.0 licence, and from its Wikidata record (CC0). EveryLLM rewrote the text in its own words, with AI assistance, and compares every page with the article to make sure no passage is copied. It may be reused under the same CC BY-SA 4.0 licence with a link back.

It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

The weekly digest

Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.