VideoPoet
A Google Research language model that turns text and images into video and audio
VideoPoet is a large language model from Google Research, announced on 19 December 2023, that generates video and audio from text prompts or still pictures. It works autoregressively and can animate a static image. The work won a Best Paper Award at ICML 2024.
- Announced by Google Research on 19 December 2023
- Generates video and sound from text prompts or still images
- Animates a static picture on request
- Built as an autoregressive language model
- Best Paper Award at ICML 2024
What it is
VideoPoet is a large language model built by Google Research to produce moving pictures. Rather than only writing text, it outputs video and sound. A user can type a description and receive a clip, or hand it a still photograph and ask for that picture to be brought to life. Text, images and existing videos all work as inputs, and the system can convert between these forms. Google Research made the project public on 19 December 2023. The paper behind it later earned a Best Paper Award at ICML 2024.
How it came about
Google Research created VideoPoet during 2023 and presented it to the public on 19 December of that year. The model was offered as a demonstration of what an autoregressive language model could do with video rather than as a shipped product. Its accompanying research paper was recognised with a Best Paper Award at the ICML 2024 conference.
Inside the model
The system is built as an autoregressive language model, meaning it predicts the next piece of output step by step in sequence, the same basic approach used for text generation. Applied to video, that sequential prediction covers moving pictures and accompanying sound. The model takes in text, still images and video, and a programme layer lets any of those inputs be turned into any of the generated output formats. Detailed figures for layers, parameter count, training data and hardware are not given here.
What it can do
VideoPoet can generate clips from written descriptions and can animate a still image so that it moves. Sample outputs shown include a dog eating popcorn at the cinema and a teddy bear in a cap, sunglasses and leather jacket playing drums, both with sound. Because it accepts images and video as well as text, it can work across input and output types. No benchmark scores are reported here.
How to use it
VideoPoet was presented as a research project from Google Research. No public weights, licence terms or hosted product are described here.
Reception and debate
The research community recognised the work with a Best Paper Award at ICML 2024. No other reception is described here.
Why it matters
VideoPoet showed that a single autoregressive language model could handle video and audio generation alongside text and images, rather than needing separate systems for each output type.
Questions people ask about VideoPoet
What does VideoPoet do?
It is a large language model that produces video and audio. You can give it a written description and get a clip back, or give it a still photograph and ask for that image to be animated. It also accepts video as input.
Who made it and when?
Google Research developed it during 2023 and announced it publicly on 19 December 2023. The project is a research effort rather than a commercial product.
How does it work?
It uses an autoregressive language model, which predicts output one piece at a time in sequence. A programme layer allows any accepted input, text, image or video, to be turned into any of the generated output formats.
Has it won any awards?
Yes. The work behind VideoPoet received a Best Paper Award at ICML 2024, the International Conference on Machine Learning.
Can I download or use it?
No public release of weights, licence terms or a hosted service is described here. It was presented as research from Google Research.
Good reading and sources
The facts on this page come from the Wikipedia article "VideoPoet", written by its contributors and used under the CC BY-SA 4.0 licence, and from its Wikidata record (CC0). EveryLLM rewrote the text in its own words, with AI assistance, and compares every page with the article to make sure no passage is copied. It may be reused under the same CC BY-SA 4.0 licence with a link back.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.