Why Perplexity Falls as Language Models Get Better, and What That Does to Detectors

Here’s a small irony at the center of AI detection. The better language models get at writing, the worse one of the oldest detection tricks works. The whole thing hinges on a single number called perplexity, and that number has been sliding downward for years.

This matters because a lot of detectors, especially the early ones, were built on an assumption that’s running out of road. Let’s walk through what perplexity is, why it falls as models improve, and what that does to the tools trying to catch those models.

Key takeaways

• Perplexity measures how *surprised* a model is by a piece of text. Lower perplexity means the text was more predictable, more expected.
• As models train on more data and get better, their own output drifts toward lower perplexity, because they’re better at choosing the words a model expects.
• Early detectors leaned on AI text having unusually low perplexity. Better models erase that edge by producing text whose perplexity overlaps with human writing.
• It’s a moving target. A detector calibrated on last year’s model can misjudge this year’s, in both directions.
• This is also why short snippets and lightly edited drafts are so hard to score, and why any single perplexity number is a weak signal, never proof.

Perplexity in one sentence

Perplexity is how surprised a model is by the next word. Feed a model some text and, at each word, it had a prediction about what was coming. If the actual word was high on its list of guesses, that word was unsurprising. String all those little surprises together and you get perplexity: low when the text matched the model’s expectations, high when it didn’t.

If you want the full mechanics with worked examples, we’ve got a plain-English breakdown of perplexity and burstiness. For this article, that one-sentence version is enough. The interesting part is what happens to that number over time.

Why better models write lower-perplexity text

Here’s the part that trips people up. Perplexity isn’t just a property of the words on the page. It’s measured *by a model*. And a model assigns low perplexity to text that looks like what it would have written.

Now think about what “a better language model” actually means. It means the model has learned the patterns of language more thoroughly. It picks the expected word more often. So when a strong model generates text, that text is, almost by definition, low-perplexity *to that model* and to models like it. The machine writes what the machine expects.

Older, weaker models were clumsier. They’d occasionally pick an odd word, take a strange turn, leave a seam showing. Those seams nudged perplexity up. As models got bigger and better trained, the seams disappeared. The output got smoother, more expected, lower-perplexity.

So the trend is mechanical, not mysterious. Improve the model, and its average output perplexity drops.

The catch: whose model is doing the measuring?

This is the detail almost every explainer skips, and it’s the crux of everything. Perplexity is always measured *by some scoring model*. There’s no universal perplexity meter floating in the air.

Detectors pick a model to do the scoring. For a while, a lot of them effectively scored against the patterns of early models like GPT-2. That worked when the text being checked also came from models in that era. The detector’s expectations and the writer’s habits were in sync.

But when the model that produced the text races ahead of the model the detector scores with, the two drift apart. Text from a 2025 model, scored by an older yardstick, doesn’t sit where the detector’s thresholds expect. That mismatch is one driver behind a lot of the confusion, including why two detectors hand you different scores for the same paragraph.

A mini-scenario

Picture two detectors and one essay.

Imagine a detector built in early 2023. Its rule of thumb, roughly, is “AI text has low perplexity; human text has high perplexity.” Back then the AI text it saw came from the models of the day, and there was a real gap. Human writing was bursty and a bit unpredictable. Machine writing was flat and smooth. The detector could draw a line between the two clusters and catch a decent chunk of AI text.

Now run text from a much better, newer model through that same detector. The newer model writes fluently, varies its phrasing, and produces text whose perplexity sits *inside* the range humans occupy. The two clusters that used to be separate now overlap. The old line doesn’t separate anything anymore.

Two failure modes follow, and they pull in opposite directions. The detector misses real AI text because it no longer looks unusually low-perplexity. And it keeps flagging the same plain human writing it always did, because that writing was always low-perplexity. So you get more misses *and* the false positives stick around. This is the practical shape of why detectors lag behind new models, and it’s why a detector that looked accurate last year can quietly degrade without anyone touching its code.

Why human writing got caught in the crossfire

If low perplexity means “probably AI,” then any human who writes in a low-perplexity way is in trouble. And plenty of people do.

Clear, plain, careful prose is low-perplexity by nature. So is writing from people who learned English as a second language and lean on common, safe phrasing because it’s reliable. A 2023 study from Stanford researchers led by Weixin Liang found AI detectors were biased against non-native English writers, and the authors point to roughly this mechanism: that writing tends to be predictable in the ways the math punishes.

So the low-perplexity-equals-machine assumption was never clean. Better models made it worse from one direction by lowering AI perplexity into human territory. The diversity of human writing always made it leaky from the other direction. Squeeze both ends and the single-number approach stops being trustworthy.

It’s worth remembering OpenAI shut down its own AI text classifier in July 2023, citing a low rate of accuracy. The company that built the model couldn’t reliably detect the model. That tells you how slippery this is.

What detectors do about it now

Serious detectors stopped betting everything on one statistic a while ago. A few things changed.

They blend features. Perplexity becomes one input among many, sitting alongside burstiness, repetition patterns, and learned signals rather than carrying the whole verdict.

They train classifiers on fresh samples. Instead of a fixed threshold, they collect new examples of human and AI writing and retrain, so the system tracks the current generation of models rather than a frozen snapshot.

They recalibrate constantly. A new model ships, output gets less surprising, the detector gets retrained on that output. It’s a genuine back-and-forth with no finish line, which is why detector accuracy numbers swing depending on which models were in the test set.

None of this makes detection reliable enough to treat a score as a verdict. It just makes it less brittle than the old single-number approach.

What this means if you write

A few practical things fall out of all this.

A score is a probability, not proof. It tells you how the text compares to a model’s expectations, and those expectations are a moving, model-specific thing. Treat any number as one signal.

Don’t chase a perplexity target. Two drafts can land on the same score and read completely differently to an actual person. The number is downstream of style, not the point of it.

Write in a way that varies. Mix sentence lengths. Drop a short line after a long one. Use a specific example instead of a smooth abstraction. That raises the texture of your writing in ways both the math and real readers respond to, without changing what you’re actually saying.

And if you’re curious about the broader picture, you can browse the rest of the blog for more on how these tools really work, or see the plans if you write at volume.

Frequently asked questions

Does a better language model always produce lower-perplexity text? On average, yes, when you measure with that same improved model. A stronger model assigns higher probability to the words it generates, which is the definition of lower perplexity. But this is about averages, not every sentence. A good model can still write a surprising line, and prompting it to be creative or terse will push individual passages up.

If AI text has low perplexity, can’t detectors just flag everything below a cutoff? That was roughly the early approach, and it’s why so many people got falsely flagged. Plenty of human writing is low-perplexity too, including clear plain prose and writing by people who learned English as a second language. A fixed cutoff catches machines and innocents alike.

Whose perplexity are we even talking about? Perplexity isn’t a property of the text itself; it’s measured by a specific scoring model. Text that looks low-perplexity to one model can look different to a newer one. When the model that wrote your text and the model the detector scores with drift apart, the numbers stop lining up.

Does this mean newer AI writing is undetectable? Not undetectable, but harder to catch with perplexity alone. That’s why modern detectors blend many features and retrain on fresh samples. The honest takeaway for writers is that any single score is a probability, not proof.

Closing

Perplexity falling as models improve isn’t a glitch. It’s the predictable result of machines getting better at sounding expected, which happens to be the exact thing old detectors measured. The number didn’t lie; the ground under it moved.

If you want to understand where your own writing actually stands, the most honest tool is one that shows you the texture instead of a verdict. Run a draft through the free checker and read the heatmap instead of fixating on a single percentage, then decide for yourself.