Top-p sampling
Selecting the next token from a small set of likely candidates that together cover a probability threshold.
Which tokens remain available for selection?
Top-p sampling, also called nucleus sampling, limits next-token candidates by their combined probability. The most likely tokens are included until the smallest corresponding set covers at least the threshold p. Sampling then uses rescaled probabilities within that set. The number of candidates can change at every step. Interaction with temperature or other filters depends on the implementation; p is not the share of business-correct answers.
A fictional calculation considers only candidates A, B and C with probabilities 0.5, 0.3 and 0.2. At p = 0.75, A and B together cover 0.8 and form the selection set; C is excluded. The remaining candidates’ probabilities are normalised to sum to one for sampling. This describes a single selection point. The distribution can look different at the next step. A business text suggestion therefore still needs checks for content, unsupported additions and the required format.
Sources
Related terms
- Prompt engineeringDeliberately designing and checking tasks, context and output rules for a generative model.
- LLM evaluationLLM evaluation tests model outputs against defined tasks and criteria. One good example does not establish overall quality.
- Confidence scoreA confidence score assesses a model prediction according to a particular method. It does not grant business approval.
- Temperature in language modelsA setting that influences the distribution of possible next tokens during sampling and therefore their selection.
- Reproducibility of AI outputsBeing able to repeat AI generation under documented conditions and compare results against defined criteria.
- Re-rankingRe-ranking reassesses search candidates already retrieved. Relevance to the query and the limits of the initial selection both matter.
Your project with ASES
Describe your project to us. We will get back to you with an initial assessment.