Top-p sampling
Sampling only from the smallest group of tokens whose probabilities add up to a set share, such as 90 per cent.
Also: nucleus sampling
Top-p, or nucleus sampling, was introduced by Ari Holtzman and colleagues in 2019 to fix the dull, repetitive text that greedy and beam search produce and the incoherent text plain sampling can produce. At each step, tokens are ranked by probability and the model samples only from the top ones that together reach the threshold p. When the model is confident, the group is small; when many words would fit, it widens. Top-p is usually combined with temperature.