Exploring the “logit_bias” Parameter in the API: How I Tackled Em Dashes and Had to Suppress 106 Tokens — My Results and Code for a “Dash-Free” Response Test

Achieving a Dash-Free Response: How Fine-Tuning Logit Bias Can Limit Em Dashes in AI Outputs

Dealing with unwanted em dashes in AI-generated text can be surprisingly challenging. Many users have struggled with models repeatedly incorporating these punctuation marks, despite various prompt adjustments and instructions. However, a powerful, albeit blunt, solution exists within OpenAI’s API parameters: logit_bias.

Understanding the Challenge

Em dashes (—) and their variants (en, em, hyphen) are often embedded within tokenization schemes, making them tricky to suppress entirely. Even when attempting custom instructions or memory management, models tend to revert to their привыч barrels of punctuation, including em dashes, especially when context encourages their use.

The Power of Logit Bias

The logit_bias parameter allows direct manipulation of token probabilities during generation. Assigning a bias score between -100 and 100 to specific tokens effectively suppresses or promotes their usage. For example, setting the bias of the em dash token to -100 causes the model to overwhelmingly avoid producing it.

My Approach

To minimize or eliminate the use of em dashes:

• First, identify the token ID associated with —.
• Next, include this token ID in the logit_bias dictionary with a bias of -100.
• Recognize that tokens can combine or be substituted by similar characters (e.g., en dashes, hyphens). Therefore, the suppression might need to extend to related tokens, sometimes requiring setting biases for dozens of tokens.

In my experiments, I found that suppressing 106 tokens related to dash characters was necessary to effectively prevent the model from using any dash-like punctuation.

The Process in Practice

Starting with minimal bias application, I incrementally increased the suppression:

• Initially, only tokens exactly representing — were biased.
• When the model persisted in using different dash forms, I expanded the list.
• Ultimately, after biasing 106 tokens, the responses no longer contained em dashes, even in models like GPT-4.

This method involved:

• Enumerating tokens with dash characters in their IDs.
• Applying a bias of -100 to these tokens.
• Testing outputs across different prompts, including complex “hot take” responses and sensitive topics.

Results and Observations

Post-bias responses showed a remarkable reduction in dash usage. Importantly, the overall quality and coherence of the responses remained intact across models, especially those considered