AI Writing Still Sounds Like AI Even When Model Bans Common Words

A developer set out to test whether logit_bias, an OpenAI API parameter that adjusts token probabilities, could make AI-generated articles sound more human by suppressing words that appear unusually often in AI prose. The approach differed from traditional prompt-based word lists: logit_bias inserts a numerical penalty directly into the model's scoring process before sampling the next token, rather than asking the model to follow an instruction it might ignore.

The mechanism works by modifying logits—the numerical scores the model assigns to every possible next token before converting them to probabilities. OpenAI's Chat Completions API allows values from -100 to 100 for each token ID. A negative value reduces the likelihood of selection. Crucially, once a substitution is forced, the new token becomes part of the context for scoring all following tokens, potentially reshaping entire sentences downstream. The developer also discovered that words don't map neatly to single tokens: " crucial" and "Crucial" may have different token IDs, and some words span multiple tokens, so penalizing one token can accidentally suppress unintended words.

To test the approach, the developer sent eight complete AI-generated articles to DeepSeek V4 Flash (routed through OpenRouter with the Cloudflare provider pinned to ensure stable support) and asked it to edit them while preserving every fact, name, number, and conclusion. Each article was processed twice: once as a control with no bias, and once with a -8 penalty applied to token IDs on a blacklist of AI-sounding words. The source articles contained 264 occurrences of watched words. Blacklist-active rewrites retained 180 of those, removing 84. Average 5-gram overlap (a measure of shared five-word sequences) dropped from 0.982 in controls to 0.896 in biased rewrites, indicating more text substitution.

When the developer tested -100 (near-total ban) in a separate hard-ban probe, the watched words disappeared entirely but the model repeated the same text until hitting the output limit—a useless result. The -8 setting proved more practical. Seven of the eight blacklist-active rewrites preserved source facts, entities, and certainty levels; one removed the hedge "appears" and changed an entity, turning a qualified observation into a direct claim. Three blacklist-active rewrites were judged better than their sources, four were equal, and one was worse (introducing ungrammatical constructions like "That details why…"). When the developer rejected any article that either changed meaning or worsened prose, six of eight blacklist-active rewrites passed, compared to seven of eight controls—a small cost.

For comparison, the developer tested a prompt-based approach with GPT-5.6 Sol (which does not support logit_bias). Five control articles used normal editing instructions; five experimental calls included the same task plus an explicit 93-word suppression list in the prompt. The prompt list reduced watched-word occurrences from 11 to one, but two of five experimental articles dropped or changed information while all five controls preserved meaning—a tradeoff the API-level method did not impose as severely. However, the developer noted this was not a direct comparison due to different models and source sets.

Support for logit_bias is uneven across providers. Every OpenAI model tested (GPT-5.2 Chat through GPT-5.6 Luna) returned an unsupported_parameter error. Anthropic's Messages API lacks the parameter entirely. OpenRouter's Models API listed 115 model IDs with logit_bias support as of July 29, 2026, including DeepSeek V4 Flash, GLM 5.2, Llama 3.3 70B, Mistral Small 3.2 24B, and Qwen 3.6 27B. Yet even OpenRouter's list is not proof of working support: some providers ignore unknown parameters, others accept but ignore the bias map, and routing rules mean that moving from one provider to another can break compatibility. The developer confirmed that DeepSeek V4 Flash worked only with the Cloudflare provider, require_parameters enabled, and fallbacks disabled.

The core insight is that logit_bias and prompt blacklists fail in opposite ways. The prompt version is flexible and can preserve meaning by recognizing context, but the model may ignore it or follow it mechanically, creating awkward text. The API version applies direct pressure to token selection without knowledge of whether a blacklisted word is actually correct in context—leading to substitutions that appear to come from a less capable model. Increasing the penalty from -4 to -12 did not consistently improve results; it only increased the risk of repetition or ungrammatical output. Neither method reliably produces human-sounding prose without some cost, suggesting that truly fixing the problem may require retraining or fine-tuning rather than token blacklisting.

The developer's experiment reveals why current approaches to scrubbing AI-sounding prose face fundamental limits. Token-level suppression via logit_bias is mathematically enforced—a -100 penalty can reliably ban a word—but the model has no way to know when the blacklisted token was the right choice. A word like "appears" might hedge an uncertain claim that should be preserved, or it might be pure filler; the API parameter cannot tell the difference. Similarly, many words tokenize in multiple forms (" crucial" versus " Crucial"), and penalizing one fragment can accidentally suppress different words. The alternative, embedding the blacklist in the prompt, gives the model discretion to ignore it when higher priorities conflict, but that flexibility comes with a cost: the model may preserve a watched word anyway, or follow the list so rigidly that it introduces awkward synonyms or damages the underlying meaning.

The test results suggest there is no clean solution. OpenAI's removal of logit_bias from current models hints that the feature may have been deemed less useful than the engineering effort required to maintain it. OpenRouter's broad support across open and hosted models offers a playground for further experimentation, but even there, provider routing rules and parameter-handling inconsistencies mean that support is not guaranteed unless the user tests each endpoint separately. The developer's findings imply that making AI writing sound human likely requires not suppressing specific words at the API level, but instead retraining or fine-tuning models to learn a different voice—a much harder problem than token blacklisting can solve.

No comments yet. Be the first to share your thoughts!