Hallucination
When a model states something false or made up, such as a fake citation, with apparent confidence.
Also: confabulation, fabrication
Language models generate plausible text, and plausible is not the same as true. They can invent facts, quotes, court cases, references and statistics, especially about obscure topics or when pushed for detail they do not have. Lawyers in the United States and elsewhere have been sanctioned for filing briefs citing cases invented by chatbots. Grounding answers in retrieved sources, training models to say when they are unsure, and checking claims reduce hallucination but have not eliminated it. Some researchers prefer the word confabulation.