UI-TARS
Open-source multimodal agent for GUI tasks
UI-TARS is an open-source multimodal agent developed by ByteDance Seed that interacts with graphical user interfaces using screenshots as input. Released in early 2025, it uses reinforcement learning to reason before acting, achieving strong results in computer use and gaming benchmarks.
- Achieved 42.5 score on OSWorld computer use benchmark
- Uses System-2 reasoning for deliberate decision making
- Open-source weights available for the 7B variant
- Perfect scores in multiple browser-based puzzle games
- Trained using reflective online traces from virtual machines
What it is
UI-TARS is a native graphical user interface agent model built by ByteDance Seed. It perceives screenshots as its primary input and performs human-like interactions, such as keyboard and mouse operations. The model operates end-to-end, meaning it does not rely on heavily wrapped commercial models or expert-crafted prompts. Instead, it directly processes visual information to execute tasks across various digital environments, including desktop operating systems, web browsers, and mobile devices.
How it came about
The foundational architecture was introduced in a paper published in January 2025. The team released UI-TARS-1.5, which integrates advanced reasoning capabilities enabled by reinforcement learning. This version allows the model to think through its steps before taking action, improving adaptability. The developers also released a 7-billion parameter variant focused on general computer use, while the larger model retains advantages in game-based scenarios.
Inside the model
The model leverages a vision-language foundation enhanced by System-2 reasoning. This approach incorporates deliberate thought processes into multi-step decision making, including task decomposition and reflection. Training involves iterative refinement using reflective online traces collected from hundreds of virtual machines. This method addresses data bottlenecks by automatically filtering and refining interaction traces, allowing the agent to learn from mistakes with minimal human intervention. Unified action modeling standardizes interactions across different platforms.
What it can do
UI-TARS demonstrates strong performance in grounding and task execution. On the OSWorld benchmark for computer use, it scored 42.5, outperforming previous state-of-the-art models. It achieved 64.2 on Android World for phone use and 75.8 on Online-Mind2web for browser tasks. In grounding evaluations, it scored 94.2 on ScreensSpot-V2. The model also excels in gaming, achieving perfect scores in several browser-based puzzles like 2048 and Hex FRVR, where other models struggled.
How to use it
The weights for UI-TARS-1.5-7B are available openly via Hugging Face. The code and desktop application are hosted on GitHub. For the top-performing UI-TARS-1.5 model, the developers provide early research access to facilitate collaborative study. Interested researchers can contact the team directly. This tiered access allows broader community engagement with the smaller model while controlling distribution of the most capable version.
Why it matters
UI-TARS represents a shift towards native agents that do not depend on external commercial large language models for core reasoning. By demonstrating that end-to-end models can outperform sophisticated frameworks built on wrapped APIs, it influences the development of autonomous GUI agents. Its use of iterative training with reflective traces offers a pathway for scaling agent capabilities without proportional increases in manual data labeling.
Questions people ask about UI-TARS
What makes UI-TARS different from other AI assistants?
Unlike assistants that rely on wrapped commercial models and complex prompts, UI-TARS is an end-to-end native agent. It perceives screenshots directly and outputs keyboard and mouse actions, allowing it to interact with graphical interfaces without intermediate translation layers.
Can UI-TARS play video games?
Yes, the model excels in playing browser-based games. It achieved perfect scores in titles such as 2048, Hex FRVR, and Infinity Loop. The full UI-TARS-1.5 model holds significant advantages in game-based scenarios compared to the smaller 7B variant.
How does the model learn to improve its actions?
It uses iterative training with reflective online traces. The system automatically collects interaction data from hundreds of virtual machines, filters it, and refines it. This allows the model to learn from its own mistakes and adapt to new situations with minimal human oversight.
Good reading and sources
There is no English Wikipedia article on UI-TARS yet, so the facts on this page come from the sources listed above: huggingface.co/ByteDance-Seed/UI-TARS-1.5-7B, arxiv.org/abs/2501.12326. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.