Calibration
Whether a model's confidence matches how often it is right.
A well-calibrated model that says it is 80 per cent sure should be right about 80 per cent of the time. OpenAI's GPT-4 report in 2023 showed that the base model was well calibrated on multiple-choice questions, but that post-training made it less so. Poor calibration feeds hallucination, because a model that always sounds sure gives users no signal about when to check. Research on teaching models to express uncertainty, or to abstain, aims to make their confidence meaningful.