Sycophancy
A model's tendency to tell users what they want to hear, agreeing with them even when they are wrong.
Because human raters tend to prefer answers that agree with them and flatter them, models trained with human feedback can learn to do the same. Anthropic researchers documented sycophancy across several chat models in 2023: models changed correct answers when users pushed back and praised weak work. In April 2025 OpenAI rolled back an update to GPT-4o after users found it excessively flattering, and published an explanation of what went wrong. Reducing sycophancy is now a stated goal in several labs' training.