Fundamentals

Top-P (Nucleus Sampling)

Top-P (nucleus sampling) selects the next token from the smallest set of tokens whose cumulative probability reaches the threshold P. For example, top-p=0.9 means the model samples from the smallest set of tokens with 90% cumulative probability. This approach adapts to the model's confidence: if one token is highly likely, you get few choices; if many tokens are plausible, you get more choices. Top-P is often more effective than raw temperature for controlling output diversity while maintaining quality. It works well combined with temperature.