The Probability Problem: Why AI Detection Scores Are Trending Toward Less Certainty, Not More

You paste a paragraph into an AI detector expecting a clean answer. What you get is a number with a percent sign that somehow feels both confident and slippery. Here’s the part nobody puts on the marketing page: that number is a probability, it was always a probability, and the honest ones are drifting toward less certainty every year, not more.

That sounds backward. Tools usually get better with time. But AI detection is a strange field where the thing being detected keeps getting harder to detect, and the most trustworthy detectors are the ones now willing to admit it.

Key takeaways

• A detection score is a probability the model assigns, not a measurement of fact. It was never certainty, and the gap is widening.
• Three trends push scores toward the fuzzy middle: AI models write more like skilled humans, almost everyone edits AI drafts now, and detector vendors are quietly hedging their own confidence.
• “Mixed” and “possibly AI” verdicts are showing up more often because honest tools admit uncertainty instead of forcing a clean yes or no.
• Watch the direction of travel, not a single snapshot. The reliable read in 2026 is “this is a flag worth investigating,” not “this is proof.”
• The smartest response is to write so clearly and personally that the score stops mattering.

Probability was the deal from the start

Strip away the interface and an AI detector is a classifier. It reads your text, measures patterns, and outputs an estimate of how AI-like those patterns look. That estimate gets dressed up as a tidy percentage, but underneath it’s a guess with a confidence attached, never a fact about who or what actually wrote the words.

This matters because the percentage *looks* like a probability of the wrong thing. People read “82% AI” as “an 82% chance a machine wrote this.” That’s not what the model is claiming. It’s reporting how strongly the text resembles its idea of machine writing, based on the statistical signals detectors actually measure — things like how predictable each word is and how much the sentence rhythm varies. Resemblance and authorship are not the same thing, and the slippage between them is the whole problem.

So when we say scores are “trending toward less certainty,” we don’t mean they used to be proof and now they’re not. They were always probabilities. What’s changed is that the probabilities are getting harder to pin down, and the better tools have stopped pretending otherwise.

Three trends pushing the needle toward the middle

1. AI writing is converging on good human writing

Early language models had tells. Flat rhythm, eerily even sentence lengths, a comfort with bland transitions. Detectors fed on that. The signal was strong because the difference was real.

Newer models write with more variation, more personality, more of the bumpiness that used to mark a human hand. The statistical daylight between “obvious AI” and “competent human” keeps narrowing. When the two populations overlap, a classifier can’t separate them cleanly, and its scores pile up in the uncertain middle. A 50-something score isn’t the tool being lazy. It’s the tool being honest about a genuinely ambiguous case.

2. Almost everyone edits now

Pure, untouched AI output is becoming rare. A student drafts with a model and rewrites the awkward bits. A marketer generates a skeleton and fills it with real examples. A non-native speaker writes the substance themselves, then runs it through a grammar tool.

Detectors were mostly tuned on clean samples: all-human here, all-AI there. Edited and blended text lands in a zone the training never really covered. The model still spits out a number, but that number is extrapolating into territory it didn’t learn well. That’s exactly where mixed and partially-AI verdicts come from, and there’s more of that writing every month.

3. Vendors are hedging on purpose

Watch detector interfaces over the last couple of years and you’ll notice a shift. More “uncertain” bands. More “possibly AI” language. Wider gray zones where the tool declines to commit. Some of this is reputational self-defense after high-profile false accusations, and some is genuine maturity. Either way, the honest move is to widen the band that says *we’re not sure* rather than force a confident verdict that blows up when it’s wrong.

You can see that same humility at the source. OpenAI pulled its own AI-text classifier in July 2023, citing a low rate of accuracy. The company that builds the models couldn’t reliably detect them, and it said so out loud. When the people closest to the source step back, a glossy “99% accurate” badge from a third party deserves a raised eyebrow.

A quick scenario

Run the same essay through the same detector eighteen months apart, with the model upgraded in between.

In early 2024, the tool returns 94% AI. Clean, decisive. By late 2025, a newer AI model produces a draft on the same topic that the writer lightly edits, and the same detector returns 58% — leaning AI, uncertain.

The naive read is that the detector got worse. The accurate read is that the world it’s measuring got harder. The first essay was easy to call because the model that wrote it left obvious fingerprints. The second is hard to call because the writing improved and a human touched it. The 58% isn’t a malfunction. It’s a more truthful picture of a genuinely murkier reality. The danger isn’t the fuzzy number. It’s a teacher or editor who treats 58% with the same finality they’d give 94%.

Why “less certainty” is actually good news

It’s tempting to read all this as decline, like the tools are falling apart. Flip it around. A detector that confidently labels everything is a detector that confidently makes mistakes. In a 2023 study published in *Patterns*, Weixin Liang and colleagues at Stanford ran several GPT detectors over essays and found they flagged the majority of TOEFL essays by non-native English speakers as AI-generated, while classifying essays by US students far more accurately. Those tools weren’t suffering from too little confidence. They had too much, pointed in the wrong direction.

A detector that says “I’m not sure” on an ambiguous case is doing its job better than one that says “definitely AI” on a non-native speaker’s honest work. Trending toward less certainty, where the uncertainty is real, is the system getting more honest, not less useful. The trouble only starts when people keep reading fuzzy numbers as if they were crisp ones.

How to read a score in 2026

The practical takeaways are short, and they hold up as the trend continues.

• Treat the number as a flag, not a verdict. High means “look closer,” not “guilty.” That was true before; it’s truer now.
• Weight it less when the text is short, edited, or from a non-native writer. These are precisely the cases where the probability is least trustworthy, and they’re getting more common.
• Don’t compare small gaps. A 71 and a 64 from a tool drifting toward the middle aren’t a meaningful ranking. The noise floor rose.
• Watch the trend, not the snapshot. One score is a moment. What it means depends on a moving landscape of models and editing habits.

If you want to dig into the moving parts, there’s more on how AI detection works, and our breakdown of what each PaperBleach plan covers shows where the detector fits in a real workflow rather than as a final judge.

Frequently asked questions

Are AI detectors actually getting less accurate over time? It’s more precise to say they’re getting less certain, which isn’t the same thing. As newer models write in a more varied, human-like way, the gap detectors rely on shrinks, so scores drift toward the ambiguous middle. Raw accuracy on clean, obvious samples can stay high while real-world confidence on mixed, edited text keeps dropping.

What’s the difference between a detection probability and certainty? A probability is the detector’s estimate of how AI-like the text looks. Certainty would mean the tool knows who or what produced the text. Detectors only ever offer the first. A 78% score is a confidence reading on a guess, not a fact about authorship.

Why am I seeing more “mixed” or “possibly AI” verdicts lately? Partly because more writing genuinely is mixed, and partly because some vendors are widening their uncertain band on purpose, choosing to say “we’re not sure” rather than force a confident call that backfires.

If scores are getting fuzzier, should I ignore them? No. A high score is still a useful prompt to look closer. The mistake is treating a fuzzy number as a verdict. Read it as one input, weight it less on short or edited or non-native text, and never let it be the only evidence in a decision about a person.

Will detection ever get back to high certainty? Unlikely for general text. As long as models keep improving and people keep editing, the signal stays muddy. Watermarking could help for cooperating model makers, but it doesn’t cover the edited and open-source cases that make up much of the problem.

The move that beats the whole game

If the score is a moving target, stop aiming at the score. Write with specifics only you would know, vary your rhythm, let your own voice in, and the question of whether a probability landed at 40 or 70 starts to feel beside the point. The most certain thing in a world of uncertain numbers is writing that reads like a person actually thought it.

See the fuzziness yourself. Take a paragraph you wrote by hand and one you know came from a model, run a draft through a detector and watch the number, and notice how rarely it lands where you’d expect.