Perplexity and Burstiness Explained: The Statistics Behind AI Detectors

Most AI detectors don’t actually “know” anything was written by a machine. They guess, using two statistics with strange names: perplexity and burstiness. Learn what those two numbers measure and a lot of the confusing stuff about AI detection, including why it flags innocent people, stops being confusing.

This is the long version. By the end you’ll know what each statistic is, how to picture the math without any math background, why human writing tends to score differently than AI, and where the whole approach quietly falls apart.

Key takeaways

• Perplexity measures how “surprised” a language model is by your next word. Predictable text scores low; surprising text scores high. AI usually writes predictably, so low perplexity reads as a flag.
• Burstiness measures how much that predictability *varies* across a piece. Humans swing between simple and complex sentences. Models often stay flat, and flat looks machine-made.
• Both are statistics, not lie detectors. A detector outputs a probability, not proof.
• The same math that catches ChatGPT also catches plain, careful human writing, which is why false positives happen.
• Even OpenAI couldn’t make this reliable, and it pulled its own detector in 2023.

Start with one question a language model asks

Strip away the branding and every AI detector is built on top of a language model doing one repetitive job: predicting the next word.

Give a model the phrase “the cat sat on the ___” and it will rank likely next words. “Mat” sits near the top. “Floor,” “couch,” and “windowsill” follow. “Differential” sits way down at the bottom because almost nobody finishes that sentence that way.

That ranking is the engine behind both of our statistics. Detectors don’t read for meaning. They check, word by word, how well their model could have predicted what you actually wrote. That’s it. Everything else is bookkeeping on top of that one move.

Perplexity: how surprised the model is

Perplexity is a measure of how predictable your text is to a language model. Low perplexity means the model saw your words coming. High perplexity means you kept surprising it.

GPTZero, one of the better-known tools, describes perplexity in its own words as “a measure of how likely an AI model would have chosen the exact same set of words as found in the document.” If a model would’ve picked the same words, perplexity is low. If your choices keep diverging from its predictions, perplexity climbs.

Here’s the leap detectors make. Large language models tend to favor high-probability next words, so their output, by default, is the stuff a model finds unsurprising. Low perplexity. When a detector sees text that its model could have generated almost word for word, it leans toward “AI.”

A worked example

Take two sentences that say roughly the same thing.

> Sentence A: “The study shows that the results are important and that more research is needed.”

> Sentence B: “The study’s real bombshell is buried in a footnote nobody’s reading.”

Walk Sentence A word by word. After “The study shows that the,” a model confidently expects “results.” After “the results are,” “important” is a top guess. “More research is needed” is one of the most predictable phrases in all of academic writing. Almost every word lands where the model expected, so perplexity stays low.

Now Sentence B. “Bombshell” is a swerve. “Buried in a footnote” is vivid and specific. “Nobody’s reading” is a casual aside a formal model wouldn’t reach for first. The model is repeatedly caught off guard, and perplexity rises.

Same basic claim, very different statistical fingerprint. A detector would treat Sentence B as more human, because it is harder to predict. That’s the whole intuition: surprise reads as humanity.

Burstiness: variation is the real tell

Perplexity alone is too crude. A human can write a flat, predictable sentence, and an AI can occasionally land a weird one. So detectors add a second layer that’s arguably more important: burstiness.

Burstiness measures how much your perplexity and sentence complexity *vary* across the whole piece. GPTZero calls it a measure of “how much writing patterns and text perplexities vary over the entire document.” Think of it as the spread, not the average.

Here’s why it matters. Human writing is lumpy. We write a long, winding sentence with three clauses, then stop with a single word. Then a medium sentence to recover. We get excited and ramble, then snap back to something terse. Plotted out, our sentence-by-sentence complexity looks like a jagged mountain range.

Machine writing tends to be smooth. Models settle into a comfortable rhythm and hold it: sentence after sentence of similar length, similar structure, similar predictability. Plotted out, it looks like a gentle, even hum. Low burstiness.

Picture two graphs

Imagine measuring the perplexity of every sentence and graphing it.

• Human draft: spiky. A peak here (that weird metaphor), a valley there (a plain transition), another peak (an unexpected example). High variance.
• AI draft: a nearly flat line hovering around “moderately predictable.” Low variance.

A detector that only looked at the *average* might score these the same. The one that looks at the *variation* sees a clear difference. That difference is burstiness, and it’s why a competent AI essay can still get flagged even when no single sentence looks robotic on its own. Each sentence is fine on its own; together they’re just too even.

If you want the practical, writer-facing version of this, we cover specific sentence structures that read as human and the broader red flags that make writing look AI. Both are really just burstiness and perplexity wearing plainer clothes.

Why human writing is usually burstier

A few reasons, none of them magic.

We have moods inside a single paragraph. We over-explain something we care about and breeze past something we don’t. We reach for an odd word because it’s the *right* word, not the most probable one. We interrupt ourselves. We use a phrase from our own life that no training corpus would predict.

A model leaning toward the likely next word doesn’t do any of that unless you push it to. Left alone, it regresses toward the average of everything it has read, which is, almost by definition, the least surprising version of any sentence. Smooth, competent, and forgettable, with low perplexity and low burstiness.

That’s also why heavily edited or “polished” AI text can sometimes look *more* AI, not less. Polishing often means sanding off the weird, specific, uneven bits. You’re flattening the exact variation that signals a human.

Where this all breaks down

If perplexity and burstiness perfectly separated humans from machines, this article would end here. They don’t, and the failures aren’t edge cases.

Plain human writing scores like AI. The statistics reward surprise and variation. But plenty of perfectly human writing is, on purpose, predictable: technical documentation, legal boilerplate, a beginner writer keeping it simple, anyone trained to write clearly and consistently. Low perplexity, low burstiness, completely human. The math can’t tell the difference, which is the root of why detectors will never hit 100% accuracy.

It penalizes non-native English writers. This is the one that should bother everyone. A 2023 study by Liang and colleagues, published in the journal *Patterns*, found that GPT detectors are biased against non-native English writers. Across seven popular detectors run on 91 TOEFL essays written by non-native speakers, the average false-positive rate was 61.3%, meaning the detectors wrongly tagged most of those essays as AI-generated, while classifying US eighth-grade essays nearly correctly. The mechanism is exactly what you’d now predict: a smaller working vocabulary and more conventional phrasing produce lower perplexity, and the detector reads low perplexity as “machine.” Here’s the part that nails the cause. When the researchers used ChatGPT to enrich the vocabulary of those same essays, the false-positive rate dropped to about 11.6%. Richer, less predictable word choice raised perplexity and the flags mostly disappeared, which tells you the tools were penalizing a linguistic background, not catching cheating. We dig into the rest of the false-positive problem in Turnitin and GPTZero separately.

Even the people who build the models gave up on it. OpenAI launched its own AI text classifier in January 2023 and quietly shut it down that July, citing a low rate of accuracy. The numbers tell the story. By OpenAI’s own published evaluation, the tool correctly caught only about 26% of AI-written text as “likely AI,” while wrongly flagging 9% of human-written text as AI. If the company with the most direct knowledge of how its model generates text couldn’t reliably detect that text, that should reset your expectations for everyone else.

The tools know this. The serious ones have moved past raw perplexity and burstiness toward trained classifiers that weigh many features at once. GPTZero now describes those two stats as just one of several components in how its detector works. That’s more sophisticated, but it doesn’t escape the core problem: every one of these systems is still measuring how predictable and how varied your text is, then outputting a *probability*. A likelihood. Not a verdict.

That distinction matters most when a score gets weaponized. If you’re a student staring at a flagged essay, the mechanism is your defense: a probability is not proof, and now you can explain exactly why the number is unreliable. Here’s how to push back when a detector gets it wrong.

What this means if your writing got flagged

First, breathe. A flag is a statistical similarity, not an accusation that holds up. You can reproduce the reasoning above and show your work.

Second, if you want writing that reads as human, to *people* and to detectors, don’t chase a number. Chase variation and specificity, which is the thing the number is trying (badly) to measure. Vary your sentence lengths on purpose. Follow a long sentence with a short one. Replace a generic claim with a concrete example only you would think of. Read it aloud and fix whatever drones. That genuinely raises burstiness because it genuinely makes the writing less uniform.

The thing to remember underneath all of it: a detector is a proxy for a real reader, and a bad one. In the end, real readers decide your credibility, not a probability bar. Write for them and the statistics tend to follow.

And if you’re curious where this cat-and-mouse goes next, we mapped out the AI-detection arms race. Short version: the stats get fancier, the false positives don’t fully disappear, and “treat the score as a signal, not a verdict” only gets more true. If you ever want to compare paid options, our plans are here.

Frequently asked questions

What is a good perplexity score to look human?

There’s no universal number, because every detector trains its own model on its own data. GPTZero’s own documentation has floated a rough rule of thumb that perplexity above about 85 is “more likely than not” from a human, but it also says plainly there is no absolute scale and that perplexity is only one part of the picture. That guideline may also be stale, since GPTZero has changed its methodology since it was published. Chasing a specific value is a losing game. Two pieces can score the same and read completely differently to an actual person. Focus on whether your sentences vary and whether your examples are specific.

Why does my own writing get flagged as AI?

Usually because it’s predictable in the ways the math rewards. Short, uniform sentences with common word choices and a steady rhythm produce low perplexity and flat burstiness, which is exactly the fingerprint detectors associate with machines. Clear, plain, careful writing trips the same wires as ChatGPT. It’s not a judgment on quality. It’s a limitation of measuring style with statistics.

Do all AI detectors use perplexity and burstiness?

Most early ones leaned heavily on them, and many still use them as a first layer. The bigger tools have added trained classifiers that look at dozens of features at once, including models tuned for academic writing. GPTZero now describes perplexity and burstiness as just one of several components. But the core idea is unchanged: they’re all measuring how predictable and how varied your text is.

Can I increase burstiness without changing my meaning?

Yes, and that’s the honest version of “humanizing.” Vary your sentence lengths deliberately. Drop a three-word sentence after a long one. Swap an abstract claim for a concrete example. Read it aloud and fix the flat spots. None of that changes what you’re saying; it changes rhythm and specificity, which is what both detectors and human readers respond to.

Is a high AI-detection score proof that text was written by AI?

No. Every reputable detector outputs a probability or likelihood, not a verdict. A high score means the text is statistically similar to what the tool expects from AI, which is correlation, not proof. People have been falsely flagged, and OpenAI retired its own classifier over exactly this unreliability. Treat any score as one signal among many.

The short version

Perplexity asks, “could a model have predicted these words?” Burstiness asks, “does the predictability bounce around the way a human’s would?” Together they’re a clever, useful, and genuinely imperfect way to guess at authorship, and a guess is all it ever is.

Want to see the idea in action on your own writing? Run a draft through our free checker and watch which sentences light up. The flagged ones are almost always the flattest and most predictable, which tells you where to add variation and specificity. Just remember the score is a signal, not a verdict, so use it to improve the writing for real readers rather than to chase a number.