Top-P.
Also called nucleus sampling — a method that selects from the smallest set of tokens whose combined probability exceeds a threshold p, adapting dynamically to the model's confidence at each step.
Top-p sampling, also called nucleus sampling, selects tokens from the smallest set whose cumulative probability reaches p. With p=0.9, the model considers only the most probable tokens that together account for 90% of the probability mass. On confident steps, this might be just 2–3 tokens; on uncertain steps, it could be hundreds.
The dynamic nature of top-p is its key advantage over top-k. When the model is highly confident about the next word (say, the article 'the' after 'dogs love to chase'), nucleus sampling with p=0.9 might restrict to just a handful of tokens. When the model has many plausible continuations, it naturally expands the candidate set.
Top-p is now the most widely used sampling method for generation tasks. Most LLM APIs default to values between 0.9 and 1.0. Values below 0.9 produce more focused text but may become repetitive. Values approaching 1.0 are nearly equivalent to no top-p filtering. Combining top-p=0.9 with temperature=0.7 is a common starting configuration for creative tasks.
Related terms.
- SamplingThe method used to select each next token from a probability distribution during text generation, controlling the randomness and creativity of model outputs.
- Top-KA sampling parameter that restricts token selection to the k most probable next tokens, discarding the long tail of unlikely options.