Chatbot Arena
A public site where people compare two anonymous models' answers, used to rank chat models by preference.
Also: LMArena, Elo rating
Chatbot Arena was started in 2023 by LMSYS, a research group from the University of California, Berkeley and other universities, and later became LMArena. Visitors type a prompt, see answers from two hidden models and vote for the better one. Votes are turned into ratings in the style of chess Elo scores. It measures what people prefer, which captures tone and helpfulness that benchmarks miss, but it can reward style and length, and a 2025 study argued that some labs had tested many private variants and published only the best.