Logit Bias.

Logit bias is a fine-grained control mechanism in language model APIs that allows developers to influence token selection by directly modifying the logits (pre-softmax scores) of specific tokens before sampling. By adding positive values to encourage certain tokens or negative values to discourage them, logit bias provides precise control over model outputs without requiring model retraining or complex prompt engineering. This technique is particularly valuable for applications requiring strict adherence to formatting rules, content filtering, or domain-specific vocabulary constraints.

The mechanism works by identifying tokens through their numerical IDs in the model's vocabulary and applying bias values (typically ranging from -100 to +100) to their logits during the generation process. Positive bias increases a token's probability of being selected, while negative bias decreases it, with extreme negative values effectively banning tokens entirely. Unlike temperature or top-p sampling which affect the overall distribution shape, logit bias targets specific tokens while preserving the relative relationships among all other tokens in the vocabulary.

Logit bias offers surgical precision for output control but requires careful implementation to avoid unintended consequences. Common applications include enforcing JSON formatting by boosting bracket and comma tokens, preventing profanity by suppressing inappropriate tokens, or encouraging technical terminology in specialized domains. However, aggressive bias can lead to repetitive or unnatural text, and developers must understand tokenization to effectively target multi-character words that may split across multiple tokens.