How AI Text Detection Works and Its Real Limitations
AI text detection tools measure statistical patterns in writing. Understanding how helps you use them effectively. AI text detectors work by analyzing statistical properties of writing that differ between human and machine-generated text. They measure perplexity, which estimates how predictable the text is based on language model probabilities. Human writing tends to have higher perplexity because people make more surprising word choices. They also measure burstiness, which captures variation in sentence length and structure. Human writing is more varied than typical AI output, and that difference is measurable. How Perplexity Works in Practice Perplexity is a score derived from a language model that estimates how likely each word is given the words preceding it. When a language model generates text, it selects words with high probability, which results in low perplexity. Human writers, by contrast, often choose words that are less predictable. A human might write that the meeting was a dumpster fire, while an AI would more likely write that the meeting was unproductive. The idiom has higher perplexity because the language model assigns it a lower probability. I tested this myself last month. I took a paragraph I wrote for a newsletter and ran it through three detectors. Two scored it at roughly 15 percent AI. The third scored it at 62 percent. The same paragraph. The difference came down to which underlying model each detector used and where they set their threshold.