Repetition Penalty.

Repetition penalty is a crucial decoding mechanism that addresses one of the most common failure modes in language model generation: repetitive text output. When generating text, language models can sometimes get stuck in loops, repeating the same words, phrases, or patterns endlessly. This parameter systematically reduces the probability of tokens that have already appeared in the generated sequence, encouraging the model to explore more diverse vocabulary and maintain coherent, varied output.

The mechanism works by applying a multiplicative penalty to the logits (pre-softmax scores) of tokens that have already been used in the current generation sequence. Typically implemented as a value greater than 1.0, the penalty divides the probability of repeated tokens, making them less likely to be selected. Some implementations distinguish between recent repetitions versus distant ones, applying stronger penalties to more recent occurrences. The penalty can be applied globally to all repeated tokens or with more sophisticated schemes that consider context and semantic similarity.

While repetition penalty effectively reduces loops and monotonous output, it requires careful tuning to avoid overcorrection. Too aggressive penalties can force models to avoid legitimate repetitions that occur naturally in coherent text, such as pronouns, articles, or thematically relevant terms. The optimal penalty value varies by model, task, and desired output style, with typical values ranging from 1.1 to 1.3. Modern implementations often combine repetition penalty with other sampling strategies like temperature and top-p sampling for more nuanced control over generation diversity.