Why AI Text Detection Has So Many False Positives and What to Do About It
AI text detectors flag human writing as machine-generated more often than people realize. Here is why and how to interpret results carefully. I have run the same human-written essay through five different AI text detectors and gotten scores ranging from two percent AI to eighty-nine percent AI. The essay was entirely mine, written before I had access to any AI tool. This experience is not unusual. AI text detection has a false positive problem that the marketing of these tools does not acknowledge, and understanding why helps you interpret detection results without over-trusting them. What Detectors Actually Measure Most detectors measure statistical properties of text, primarily perplexity and burstiness. Perplexity estimates how predictable each word is given the preceding words, based on a language model. Low perplexity means the text is highly predictable, which is typical of AI output. Burstiness measures variation in sentence length and structure. Human writing tends to be bursty, with short sentences mixed with long ones, while AI output tends to be more uniform. The problem is that many forms of human writing are also low-perplexity and low-burstiness. Technical documentation, legal writing, academic prose in certain fields, and writing by non-native speakers who carefully follow learned patterns all tend to score as AI-generated. These styles are statistically regular because regularity is a feature of the genre, not because a machine wrote them. The detector has no way to distinguish regularity of style from regularity of authorship.