Why Two AI Detectors Give Different Scores for the Same Paragraph

You paste one paragraph into a detector and it reads 18% AI. You paste the exact same words into a second tool and it reads 91%. Same sentences, two machines, two stories that can’t both be true at face value. So which one is lying?

Neither, it turns out. The gap is what you’d expect once you know what these tools are actually doing under the hood. Here’s the mechanical reason two detectors look at one paragraph and see different things.

Key takeaways

• Two detectors disagree because they’re separate models trained on different text, with different scoring math and different cutoff lines. The gap is built in, not a glitch.
• A score is a probability, not a reading off a dial. 18% and 91% are two guesses about the same paragraph, not one right answer and one wrong one.
• The same tool can change its own answer if you paste a paragraph instead of the full document, or re-run it weeks later after a model update.
• Disagreement is loudest on exactly the hard cases: short passages, plain or formal prose, and writing by non-native English speakers.
• A wide split is useful information. It tells you the text sits in an ambiguous zone where no single number should be treated as a fact.

A detector isn’t a scanner. It’s a guesser.

The mental model that trips people up is imagining the detector as a barcode reader: point it at text, and it reads off some hidden “AI-ness” that’s physically printed there. That’s not what happens. There’s no watermark in ordinary AI text, no fingerprint baked into the words. The tool is making a statistical guess about how the writing was probably produced.

A better picture is two weather forecasters. They look at the same sky and one says 30% chance of rain, the other says 75%. Neither is wrong, exactly. They’re running different models that weight the inputs differently. AI detectors split for the same reason, and the split comes down to three concrete differences in how they’re built.

Reason one: they learned from different examples

Every detector is a model that was shown a giant pile of writing labeled “human” and “AI,” and learned to tell them apart. The problem is that no two vendors used the same pile.

One tool might have trained heavily on GPT-3.5 output from 2023. Another might include newer models, more academic essays, more marketing copy, or more code. A model can only recognize the patterns it was shown. Feed it something that looks unlike its training set, and it falls back to guessing, sometimes loudly and wrongly.

This is why a paragraph of clean, formal prose can split two tools so hard. If one detector saw lots of careful human academic writing during training, it learned that flat, tidy sentences can still be human. If the other mostly saw chatty human samples, that same tidy paragraph looks alien, and “alien” is the exact thing it learned to flag as AI.

Reason two: they cut the text up differently

Detectors don’t all judge a paragraph the same way. Some score it as one block. Some break it into sentences and average the results. Some slide a window across overlapping chunks. Some weight the opening more than the body.

Tokenization matters too. Before any math runs, the text gets split into tokens, and different models split it differently. The number of tokens a detector sees changes how confident it can be. This is also why pasting one paragraph versus the full essay can give you different answers from the *same* tool. The underlying statistics these tools actually run — how predictable the words are, how much the sentence rhythm varies — get computed over different amounts of material, so the numbers shift.

Short text is the worst case here. Two sentences barely give a detector enough signal to do anything but flip a weighted coin. Less text, less to measure, jumpier results, wider disagreements.

Reason three: they draw the line in different places

Here’s the one most people never think about. Internally, a detector doesn’t produce “AI” or “human.” It produces a number on a sliding scale, say 0.0 to 1.0. Then it converts that raw number into a label or a percentage you can read. *Where* it puts the cutoff is a design and business decision, not a law of nature.

Picture a paragraph that lands at 0.62 internally in both tools. Tool A calls anything above 0.5 “likely AI” and shows you a scary high number. Tool B, tuned to avoid embarrassing false accusations, waits until 0.8 before it gets alarmed, so it reports the same paragraph as probably human. Nearly identical underlying math, opposite-sounding verdicts. The difference on screen is mostly about where each company decided to draw its line.

There’s also plain instability over time. Vendors update their models quietly. A score you got last month may not reproduce this month, not because your writing changed but because the model did.

A worked example: one paragraph, two verdicts

The numbers here are illustrative, not measured results from a specific tool. They show the kind of split this setup produces. Take a plausible paragraph from a student’s essay on the Treaty of Versailles:

> The Treaty of Versailles imposed severe penalties on Germany. It required large reparations payments. These payments damaged the German economy. This economic damage contributed to political instability. The instability later helped extremist movements gain support.

Every sentence is true. Say the student wrote every word. And it’s a near-perfect trap.

A detector that leans hard on predictability and flat rhythm would likely score this high. Look at the paragraph: five sentences, almost the same length, each starting where the last one ended, no surprises. That uniformity is the same statistical signature these tools learned to associate with machine output. A second detector, trained on a broader set and tuned more cautiously, might score the very same lines as probably human, because it saw enough plain, repetitive human writing in training not to panic.

Notice what the disagreement is telling you. It isn’t noise. It points straight at a known weak spot. A 2023 Stanford study led by Weixin Liang found that several popular detectors flagged more than half of non-native English (TOEFL) essays as AI, while almost never misfiring on native-English samples. Flat, careful, “textbook” prose is exactly where false positives live, and exactly where two tools are most likely to split. The gap is a warning light, not a riddle to solve by picking a side.

What the disagreement actually means

The instinct is to ask “which detector is more accurate” and trust that one. It’s the wrong question, for a couple of reasons.

Accuracy claims come from each vendor testing on its own data under its own conditions. Those numbers rarely survive contact with your actual writing, which is a messy mix of lengths, subjects, and styles no test set matches. Independent comparisons keep finding that the “best” tool shifts depending on what you feed it.

And there’s a hard ceiling worth stating plainly: these tools output probabilities, not proof. Reputable vendors say so in their own documentation. OpenAI went further and pulled its own AI Text Classifier in July 2023 because it wasn’t accurate enough to be useful. A 91% isn’t a confession. It’s a model saying “this resembles things I was told are AI.” Resemblance isn’t authorship. If you’re weighing tools or trying to read your own results responsibly, our plans and what each check reports spell out what a score does and doesn’t claim.

So a wide split shouldn’t raise your confidence in the scary number. It should lower your confidence in *both* numbers. The healthiest read is this: the passage is hard to classify, so don’t classify it on a percentage. To go deeper on the responsible-use side, you can browse the full library of detection guides.

If it’s your own writing that’s splitting the tools

Maybe you’re not judging someone else. Maybe it’s your draft getting 20% in one checker and 80% in another, and you just want it to read like a person wrote it, because a person did.

The fix isn’t chasing a magic score. It’s the same thing that makes writing good for human readers: vary your sentence length on purpose, drop a short punchy line after a long one, swap an abstract claim for a concrete example, and read it aloud to catch the spots that drone. That’s what closes the gap between detectors, because you’re raising the variation and specificity that both the math and real readers respond to.

Frequently asked questions

Why do AI detectors disagree on the same paragraph? Each detector is a separate model trained on its own mix of human and AI writing, running its own math and its own cutoff for what counts as “AI.” Those three things rarely line up between vendors, so the same paragraph can land at 18% in one tool and 91% in another. Both numbers are estimates from different machinery. The disagreement is structural, not a bug.

If one detector says 90% and another says 10%, which is right? Possibly neither, in the way you mean. Each number is a probability that the text resembles what that model was trained to call AI, not a measurement of who wrote it. When they split that far apart, the honest read is that the paragraph is ambiguous and shouldn’t be judged on a number at all.

Why does the same detector give me a different score the second time? Some tools quietly update their models, so last month’s score may not reproduce today. Detectors are also sensitive to how text is chunked, so scoring one paragraph versus a whole document runs the math over different amounts of material. And short passages give too few tokens to be confident, which makes results jumpy.

The fastest way to stop guessing is to see it on your own text. Run your paragraph through PaperBleach and watch which sentences a detector lights up, and why, instead of staring at one mystery percentage. Once you’ve seen the patterns, you’ll never read a single score the same way again.