What a Perturbation Curve Looks Like for Human vs AI Writing

Ask an AI detector how it “knows” a paragraph came from a machine, and one of the more interesting answers is a shape. Not a number, not a verdict, but a curve. Picture taking your sentence, making fifty slightly reworded copies, scoring each one, and plotting how the scores fall away from the original. The way that line bends turns out to look very different for AI writing than for human writing, and that difference is the whole idea.

Key takeaways

• A perturbation curve plots how a passage’s model-assigned probability changes when you make many small reworded copies of it.
• AI-generated text tends to sit at a local peak: almost every reworded version scores lower, so the curve bends sharply downward.
• Human writing usually sits on a flatter, bumpier slope. Some reworded versions score higher, some lower, so the curve doesn’t dip cleanly.
• This “curvature” idea powers zero-shot detectors like DetectGPT, which need no training examples of AI text to guess.
• The method is fragile. Short passages, heavy editing, paraphrasing, and newer models all flatten the signal.

What a perturbation curve actually is

Start with one fact about how language models work: they assign a probability to text. Feed a model a sentence and it can tell you, roughly, how likely that exact string of words is according to everything it learned. High probability means “this looks like the kind of thing I’d write.” Low probability means “this surprises me.”

A perturbation curve takes that idea one step further. Instead of scoring your text once, a detector scores it many times, each time with a tiny change. It might swap a word for a synonym, reorder a clause, or mask a token and let a second model fill it back in. Each of these “perturbed” versions is a near-neighbor of your original. Then it scores them all and asks a simple question: compared to its reworded neighbors, is the original a peak, a valley, or just part of the slope?

Plot the original in the center and the perturbations around it, and you get a curve. The technical name for what the detector measures is *curvature*. How sharply does probability drop as you move away from the original phrasing?

The AI curve: a sharp peak

Here’s why this works. When a model generates text, it tends to pick high-probability words at each step. The finished passage isn’t random; it sits right around the model’s preferred way of saying things. It’s a local maximum.

So when a detector perturbs AI text, almost every change makes things worse from the model’s point of view. Swap a word, and you’ve moved off the peak. Reorder a clause, and the probability drops. Do this fifty times and you get the same answer fifty times: the original was more likely than its neighbors.

On a plot, that’s a clean, steep downward curve. The original sits at the top, and the cloud of perturbations sits noticeably below it. Strong negative curvature. The 2023 DetectGPT paper, by Eric Mitchell and colleagues at Stanford, formalized exactly this: machine-generated text tends to occupy regions of negative curvature in a model’s log-probability function, and human text doesn’t, at least not as reliably.

The human curve: a bumpy slope

Human writers don’t optimize for probability. We pick the odd word, bury the lede, write a fragment, reach for the specific noun instead of the common one. Our sentences don’t sit on the model’s peak; they sit somewhere on the messy hillside around it.

That changes the curve. When a detector perturbs human writing, the results scatter. Some reworded versions actually score *higher* than your original, because the model would have phrased it more predictably than you did. Others score lower. The neighbors land above and below, not consistently below.

On a plot, that’s a flatter, noisier line. No clean peak, no steep drop. The curvature is weak. That ambiguity, that lack of a sharp dip, is what the detector reads as “probably human.”

A mini-scenario you can picture

Two sentences mean the same thing.

The AI one: “The study demonstrates that the intervention had a significant positive effect on student outcomes.”

The human one: “The study found the program helped, though the gains were smaller for kids who started behind.”

Now imagine a detector rewording each twenty times. For the AI sentence, almost every variant nudges the probability down, because the original was already the smooth, expected phrasing. Swapping “demonstrates” for “shows” or “significant” for “notable” just moves off the peak. Result: a steep curve.

For the human sentence, the variants bounce around. A model might rate “the gains were smaller for students with lower baseline scores” as *more* probable than “kids who started behind,” because that’s the tidier, more expected academic phrasing. So one perturbation goes up, another goes down, and the curve refuses to form a clean peak. Result: a flat, ambiguous line.

Same meaning. Opposite curve shapes. That’s the signal in a nutshell. If you want the companion view, our perplexity and burstiness guide covers the single-number cousins of this idea.

Why detectors bother with this approach

The appealing thing about the perturbation method is that it’s *zero-shot*. It doesn’t need a labeled training set of “here are 10,000 AI essays and 10,000 human ones.” It just needs a language model to do the scoring. That makes it cheap to point at new domains and, in theory, less tied to one writing style than a trained classifier.

It also tries to be more stable than raw perplexity. Perplexity alone asks “how surprised is the model by this exact text?” and gets thrown off by topic, length, and the writer’s vocabulary. Curvature asks a relative question instead: forget the absolute score, is this a peak compared to its neighbors? That framing was meant to be more robust.

“More robust” is not “robust,” though.

Where the curve breaks down

This is the part the marketing skips. The perturbation curve is a clue, and a fragile one.

Short text flattens it. With only a sentence or two, there isn’t enough material to perturb meaningfully. The cloud of neighbors is tiny and the curvature estimate gets noisy. Detectors are weakest on exactly the short passages people most often paste in.

Editing reshapes it. This is the honest core. The whole AI signal depends on your text sitting at the model’s peak. The moment you genuinely revise, add a specific detail, break the rhythm, choose a word the model wouldn’t, you step off that peak. Now some perturbations score higher than your version, the downward curve flattens, and the tell fades. That’s not a hack; it’s what real editing does to the statistics.

Newer models flatten it too. The method assumes you can score the text with the same model that wrote it, or a close stand-in. As models multiply and learn to produce more varied, less peaky output, the clean negative curvature gets shallow. A detector built around one model’s probability landscape doesn’t map neatly onto text from a different one.

It measures style, not authorship. A careful human who writes in smooth, predictable prose can produce a peaky curve and get flagged. A 2023 Stanford study found detectors disproportionately flag non-native English writers, whose phrasing can read as more “average.” The curve sees predictability, not who typed it. And the broader lesson holds: OpenAI pulled its own AI text classifier in July 2023, citing low accuracy. These tools output probabilities, never proof.

How to read your own score with this in mind

You won’t usually see the actual curve. Most tools run the math quietly and hand you a percentage or a highlighted heatmap instead. But knowing the shape behind the number changes how you read it.

When a score drops after you edit, it’s often because you knocked the text off its peak: your changes made some reworded versions more probable than your own, and the curvature collapsed. When a score barely moves, you probably edited the surface, a synonym here, a comma there, without leaving the peak. That’s the practical difference between paraphrasing and actually rewriting.

Don’t chase a magic curvature value. Two passages can share a curve and read completely differently to a human. The goal is writing that genuinely doesn’t sit at the model’s default, because it’s specific, varied, and yours.

Frequently asked questions

What is a perturbation curve in AI detection? It’s a way of visualizing how confident a language model is about a piece of text relative to slightly altered versions of it. A detector rewords your passage dozens of times, scores each copy, and plots those scores against the original. How steeply the curve dips from the original is the signal.

How is this different from perplexity? Perplexity is one number: how surprised a model is by your text as written. A perturbation curve is a relationship between your text and many reworded neighbors, measured as curvature. Perplexity asks “how likely is this sentence?” The curve asks “is this sentence a peak compared to everything nearby?”

Can editing change the shape of the curve? Yes. If you rewrite AI output so it no longer sits at the model’s preferred phrasing, the original stops being a local peak, neighbors start scoring higher, and the downward curve flattens. That’s what real revision does, not a trick.

Do real detectors actually show me a curve? Usually not. They run the math behind the scenes and hand you a probability or a heatmap. The curve is the underlying mechanic for zero-shot detectors like DetectGPT.

Why does the method get weaker on newer models? It assumes you can score text with the model that wrote it or a close stand-in. As models get better at varied, less peaky output, the clean downward curve gets shallow and hard to read, and short passages make it worse.

The takeaway

A perturbation curve is just a picture of one question: compared to all the ways you could have said this, is your phrasing the model’s favorite? AI text says yes with a sharp peak. Human writing shrugs with a flat, bumpy line. Understanding that shape is more useful than any single percentage, because it tells you *why* editing moves a score and why no score is the final word.

Want to see how your own draft reads to a detector, and watch what happens when you revise it for real? Run a draft through the free checker, and when you’re ready for heavier use, see the plans.