Perplexity
A measure of how well a model predicts a text. Lower means the model was less surprised.
Perplexity is e raised to the average cross-entropy loss over a text. It can be read as the number of choices the model was, in effect, torn between at each token: a perplexity of 10 means it was as uncertain as if picking evenly from 10 options. It is the standard way to compare language models during pre-training. It does not directly measure helpfulness, truthfulness or reasoning, and it can only be compared between models that use the same tokenizer.