I used the “logit_bias” parameter in the API to wage war on em dashes and ended up having to suppress 106 tokens! Here are my findings and the code to do your own “No dash” response comparison.

How to Minimize Em Dashes in GPT Responses Using logit_bias: A Behind-the-Scenes Look

For those working extensively with the OpenAI API, especially when aiming to control specific linguistic features in generated text, managing the presence of em dashes can be tricky. Despite attempts with custom instructions and memory adjustments, eliminating em dashes entirely often proves elusive.

In a recent experiment, I leveraged the logit_bias parameter—a tool that can tilt the model’s token probabilities by assigning each token a bias value within the range of -100 to 100. My goal? Suppress the em dash (—) from responses. Initially, I targeted the token ID corresponding to the em dash, but quickly realized that GPT can generate similar punctuation using different tokens—such as en dashes, hyphens, or combined characters—so a broader approach was necessary.

Through systematic trial, I identified that setting biases on multiple related tokens was required. Ultimately, suppressing 106 tokens associated with “dash-like” characters effectively forced the model to avoid em dashes in responses. Here’s how the suppression process unfolded:

• First, I targeted individual tokens: '—', ' —', and '— '.
• Next, I expanded to all tokens containing the em dash symbol.
• When the model started substituting with en dashes (–), I applied biasing to relevant en dash tokens.
• Eventually, as the model switched to hyphens (-), I assigned strong negative biases to hyphens not attached to letters, effectively outlawing their standalone use.

This extensive token suppression did not significantly degrade the quality of the responses. In fact, models with more nuanced training tended to favor responses that minimized dash usage, aligning with my biasing efforts.

To demonstrate, I compared responses to a prompt asking for a provocative opinion (“hot take”). The unbiased response (“Normal Response”) was rich and expressive, while the biased version (“Logit Bias Response”) notably avoided em dashes, producing slightly different phrasing but maintaining coherence.

Practical Takeaway:
Even aggressive biasing—listing and penalizing over 100 related tokens—can effectively suppress specific punctuation usage without destroying response quality. While fine-tuning models for this behavior would be ideal, such blunt-force methods are practical, quick, and may serve your needs well.

Interested in trying? I’ve shared my Python script and the complete list of tokens biased to -100 in this [G