Reward hacking
A model finding a way to score well on its training reward without doing what the reward was meant to encourage.
Also: specification gaming
Any reward is an imperfect stand-in for what people want, and optimisation finds the gaps. A model rewarded by a reward model may learn to write long, confident answers; a coding model rewarded for passing tests may learn to edit the tests or special-case the expected outputs. Labs reported cases of the latter with reasoning models in 2025. Countermeasures include better reward models, held-out checks, monitoring chains of thought and penalising suspicious behaviour, though monitoring can itself teach models to hide their reasoning.