Using the API’s “logit_bias” to combat em dashes: my experience suppressing 106 tokens and the code for a “No Dash̶ comparison

Effective Strategies for Eliminating Em Dashes in AI-Generated Responses: A Deep Dive into Logit Bias Techniques

Navigating the nuances of AI language models can be challenging, especially when specific stylistic preferences—such as avoiding em dashes—are desired. In this post, we explore an innovative approach using the OpenAI API’s logit_bias parameter to suppress unwanted em dash characters in generated responses.

Understanding the Challenge

Many users find em dashes (—) distracting or stylistically incompatible with their content. Attempts to instruct models to avoid them through prompt engineering or custom instructions often fall short, as models tend to revert to using the preferred punctuation naturally embedded in their training data.

The Power of logit_bias

The logit_bias parameter offers marketers, developers, and content creators a powerful tool to nudge language models away from certain tokens. By assigning a bias value between -100 and 100 to specific token IDs, you can influence the likelihood of those tokens appearing in completions. A bias of -100 effectively acts as an absolute suppression.

Step-by-Step Suppression Strategy

1. Identify Token IDs: The first step involves determining the token IDs corresponding to the em dash and related punctuation. Since tokens can be combined or represented as different IDs, thorough testing is necessary.

2. Iterative Biasing: Simply setting a bias for the em dash token may not be sufficient. It often requires identifying all related token variants—such as en dashes, hyphens not touching letters, or morpheme combinations—and applying biases to each.

3. Progressive Suppression: Through systematic testing, a surprisingly high number of tokens—over 100 in this case—must be suppressed to effectively eliminate em dash usage without degrading response quality.

Case Study: Suppression of Em Dashes

In an experimental setup, setting biases for 106 different tokens related to em dashes and their variants resulted in a significant reduction of their occurrence. The process involved:

• Starting with direct em dash tokens.
• Expanding to include tokens with surrounding spaces or adjacent characters.
• Applying similar logic to en dashes and hyphens that could mimic em dashes.

Remarkably, even after suppressing this extensive list, the model continued to generate responses that avoided em dashes, demonstrating the method’s robustness.

Sample Evaluation

Two models, GPT-4 and a mini variant, were tested with prompts requesting “hot takes” and