Why Short Text Breaks AI Detectors: The Statistics of Too Few Tokens

A two-sentence reply comes back 92% AI. You paste the exact same two sentences into a different tool and it says 8%. Neither tool is broken. You’ve just hit the wall every AI detector runs into when the text gets short: there aren’t enough tokens to do honest statistics on.

This is one of the most misunderstood corners of AI detection. People treat a percentage on a short blurb as a verdict, when it’s closer to flipping a weighted coin. Here’s what’s actually happening under the hood, and why a “minimum word count” isn’t a bureaucratic limit but a mathematical one.

Key takeaways

• AI detectors average a statistical signal across your tokens (roughly, word pieces). Too few tokens means a noisy average that swings hard from tiny changes.
• Most major detectors set a minimum input length on purpose. GPTZero treats around 250 words as the point where accuracy holds up; Turnitin won’t score a document at all under 300 words of long-form prose.
• On short text, one odd word or one formulaic phrase makes up a big chunk of the sample, so it can flip the whole verdict.
• A short-text score is a coin flip dressed up as a percentage. Anything under a couple hundred words should be read as inconclusive.
• Want a meaningful read? Give the detector more text, or judge short passages by reading them rather than trusting a number.

What a detector is actually measuring

Strip away the marketing and most detectors do something simple in spirit. They run your text through a language model and ask, token by token, “how surprised is the model to see this word here?” Predictable next-words produce low surprise; unexpected ones produce high surprise. Average that surprise across the whole passage and you get the core signal behind scores like perplexity. (If you want the full breakdown, we covered the statistics behind AI detectors separately.)

The key word is *average*. A detector isn’t reading meaning. It’s computing a statistic over a sample, and like any statistic, it’s only as stable as the sample is big.

A “token” isn’t quite a word, by the way. Models chop text into sub-word pieces, so “humanizer” might be two or three tokens and “the” is one. A 50-word sentence might run 65 to 70 tokens. That detail matters because the detector’s math runs per token, and short text means few tokens to work with.

The statistics of “too few”

Think about flipping a fair coin. Flip it 10 times and getting 8 heads wouldn’t shock anyone; small samples are streaky. Flip it 1,000 times and you’ll land very close to 500. The more flips you take, the more the noise averages out and the closer you get to the real rate. That’s the law of large numbers, and it’s the whole story here.

Token-level surprise behaves the same way. Across 400 words, a handful of predictable phrases and a handful of surprising ones balance out into a stable average that genuinely reflects how the text reads. Across 15 words, you don’t have enough data points to balance anything. One common phrase drags the average down; one quirky word yanks it up. The “score” is real arithmetic, but the margin of error around it is enormous.

There’s a second problem stacked on top. The variation in surprise from sentence to sentence, sometimes called burstiness, is part of what separates human rhythm from machine smoothness. You can’t measure variation across sentences if you’ve only got one or two sentences. So the feature that helps tell people from machines goes dark exactly when the text is short.

A quick worked example

Say a detector flags any token whose surprise falls below some cutoff as “AI-like,” and scores the passage by the fraction of flagged tokens.

In a 600-word essay, suppose 90 tokens out of 800 fall below the cutoff. That’s around 11%. Now change five words. Maybe 95 tokens trip the cutoff, maybe 85. The percentage barely moves. The verdict is stable.

In a 12-word sentence (call it 16 tokens), suppose 6 fall below the cutoff. That’s 38%. Change one word and now 8 tokens trip it: 50%. Change two and you’re back down to 25%. Same writer, same idea, wild score swings, all because each token is worth about 6% of the total instead of a fraction of a percent. That’s not the detector being dumb. That’s small-sample math doing exactly what small-sample math does.

Why the tools set a minimum word count

Once you see the statistics, the input limits make sense. They’re guardrails against reporting confident nonsense.

• GPTZero treats around 250 words as the point where its accuracy holds up and warns that shorter text is less reliable. Its hard floor is much lower, roughly 250 characters, but that’s the bare minimum to run, not the point where the result means much.
• Turnitin’s AI indicator won’t return a score until a document has at least 300 words of long-form prose, and it leaves short or list-heavy documents unscored rather than guessing. Turnitin raised that floor from 150 to 300 words because accuracy climbs with more text.
• Plenty of smaller tools simply refuse input under roughly 50 to 100 words, or stamp the result with a low-confidence flag.

These aren’t arbitrary. They’re the vendors admitting, in product form, that below a certain sample size the honest answer is “we don’t know.” It’s worth remembering that even OpenAI pulled its own AI text classifier in July 2023, citing a low rate of accuracy overall, and short text is where accuracy collapses fastest.

The short-text trap in real life

Here’s where this bites real people.

A student’s discussion-board post. Many forum prompts ask for “a 3 to 4 sentence response.” That’s exactly the length where detectors are least trustworthy, yet it’s also the format some instructors paste into a checker. A clear, plain answer to a narrow question is short, predictable, and low-surprise by nature, the same profile a model produces. Short plus plain is the worst-case input for a fair read.

A freelancer’s product blurb. A 40-word product description has almost no room for sentence-length variety. There’s nowhere to put a long sentence next to a punchy one, so burstiness flatlines and the detector reads “machine.” The writer didn’t do anything wrong; the format forbids the very variety detectors reward.

A non-native English writer’s short answer. This compounds a known bias. A 2023 Stanford study by Liang and colleagues, published in *Patterns*, found that detectors misclassified more than half of non-native English writing samples as AI-generated while staying nearly perfect on native-English text. The authors pointed to simpler, more uniform vocabulary reading as low-perplexity. On a short answer, there’s no length to dilute that effect, so the misfire rate climbs.

In all three cases the fix isn’t to game anything. It’s to recognize that a number on a tiny sample isn’t evidence of much.

How to read a short-text score the right way

A few honest habits:

1. Check the length first. Under ~200 words, treat any score as inconclusive by default. The tool may even tell you so if you read the fine print.
2. Don’t compare percentages across tools on short text. Disagreement is the expected outcome, not a tiebreaker. Two tools saying 90% and 8% on the same blurb tells you the sample is too small, not that one of them is right.
3. Add context if you can. If the work allows it, paste a longer version, the full document the snippet came from, so the average has something to stabilize on.
4. Read it. For genuinely short passages, your own judgment about whether something sounds canned beats a noisy statistic. Real readers, not detectors, are who ultimately decide whether writing lands.

If you want to sanity-check a longer piece, you can run it through the free checker and look at the sentence-level view instead of fixating on one top-line number. For more on the mechanics and the editing side, browse the rest of our writing guides, and if you’re checking work at volume you can see plans.

Frequently asked questions

What is the minimum word count for an AI detector to be accurate?

There’s no single magic number, but the working floor most tools point to is a few hundred words. GPTZero treats roughly 250 words as the point where its accuracy holds up, and its hard minimum is far lower, around 250 characters. Turnitin won’t return an AI score at all until a document has at least 300 words of long-form prose. Below those floors, the statistics don’t have enough samples to average over, so the score gets noisy. As a rule of thumb, treat anything under about 200 to 300 words as inconclusive.

Why does the same short paragraph get different scores on different detectors?

Because each detector trains its own model on its own data and sets its own threshold, and short text amplifies those differences. With only a handful of tokens to average, one tool’s model might fixate on a phrase another ignores. The smaller the sample, the more each tool’s individual quirks dominate, so disagreement is expected, not a sign anyone found the truth.

Can I trust a 90% AI score on a two-sentence answer?

No. A high percentage on two sentences is the detector’s confidence about a sample far too small to be confident about. One predictable phrase can push a short passage to 90% as easily as it can drop it to 10%. The number looks precise, but the margin of error on short text is enormous. Detectors output probabilities, not proof, and that caveat matters most when the input is tiny.

Does adding more text always make detection more reliable?

More text gives the detector more samples to average, which cuts random noise, so longer inputs are generally more stable. But length doesn’t fix bias. If your natural style is plain and uniform, a longer sample can still read as predictable, and the Stanford study above found non-native English writers flagged at much higher rates regardless of length. Length helps with statistical noise, not with the method’s underlying limits.

Why do tools like Turnitin sometimes skip short submissions entirely?

Because they know short text produces unreliable results, and reporting a confident-looking score on it would do more harm than good. Turnitin’s AI indicator requires at least 300 words of long-form prose before it returns a score and leaves shorter or list-heavy documents unscored. That’s not a bug. It’s an acknowledgment that the math needs a sample size.

The bottom line

Short text doesn’t break AI detectors because the detectors are bad. It breaks them because you can’t do trustworthy statistics on a handful of tokens, any more than you can read a coin’s fairness from three flips. When the sample is tiny, the percentage is theater. Give the math enough text to work with, or judge the writing with your own eyes, but don’t hand a high-stakes decision to a number computed on 15 words.