Experimenting with the API’s “logit_bias” Parameter to Reduce Em Dashes: My Experience Suppressing 106 Tokens and a “No Dash̶ Response Test with Sample Code

How I Reduced Em Dash Usage in GPT Responses Using the Logit Bias Parameter

Dealing with unwanted em dashes in AI-generated text can be a frustrating challenge. Recently, I explored a novel approach to suppress their presence by leveraging the OpenAI API’s logit_bias parameter—a lesser-known feature that allows for fine-tuning token probabilities. Here’s a detailed account of my experimentation, the methodology, and practical code you can adapt for your own projects.

The Challenge with Em Dashes

Many users, including myself, have struggled with GPT models inserting em dashes (—) unexpectedly. Whether for stylistic reasons or output consistency, controlling their appearance can be tricky. Traditional methods—like setting custom instructions or adjusting memory—often fall short of completely eliminating em dashes.

Leveraging logit_bias for Token Suppression

The core idea I employed was to assign a strong negative bias (-100) to tokens associated with em dashes and their variants. The logit_bias parameter in the OpenAI API enables this by adjusting the likelihood of specific tokens during generation.

Step-by-Step Process

1. Identify Em Dash Tokens:
   The primary token for the em dash character (—) might be straightforward, but tokens can also arise through combination with surrounding characters or via similar symbols like en dashes (–) and hyphens (-).

2. Token Variants and Combinations:
   Since tokens can be joined with other characters, I systematically captured all tokens containing or related to dashes. This included:

3. Exact match for —
4. Variants touching other characters

5. En dashes (–) and hyphens (-) used in place of em dashes

6. Incremental Suppression:
   I initially set biases on the exact token for —, but GPT still produced em dashes. To eradicate their usage, I progressively broadened the suppression to include all related tokens:

7. First, tokens with any occurrence of —
8. Then, tokens with hyphens used as dashes
9. Finally, all tokens that could possibly resemble or substitute an em dash

It took setting 106 tokens to -100 to effectively suppress em dashes, highlighting how intricate tokenization can be.

The Results

Here’s a quick comparison from test runs:

Standard Response

Here’s a hot take: The