AI detectors wildly inconsistent at spotting human writing

What happened

Pangram and Grammarly correctly identified every human-written text as human in a test by the Authors Guild using ten articles published between 2020 and 2022. Originality.ai also performed well. By contrast, Sidekicker flagged every single article as mostly AI-generated, with two scoring 100 percent, and ZeroGPT reported sometimes high AI percentages for all the human-written texts.

Why it matters

False positives from unreliable detectors can cost authors their contracts and reputations. The Authors Guild warns that even the best-performing tools should never be the sole basis for any decision, and publishers should disclose their methods and give authors a chance to defend themselves. The core problem: professional writers and language models produce statistically similar text because models were trained on human-written material, making detection nearly impossible for the best-performing tools.

What to watch

The test results apply mainly to correctly recognizing human writing—Pangram and Originality.ai are tuned to minimize false positives—but do not necessarily show how well they catch AI-generated texts. The Authors Guild notes that detection tools change constantly and their accuracy cannot be taken for granted.