Cross-Model Detection: Can a Detector Trained on GPT-4 Spot Claude or Gemini Text?

Say a school buys an AI detector that was tuned mostly on essays written by GPT-4. A student turns in a draft they wrote with Claude, and another submits one from Gemini. Does the detector still light up, or does it shrug because it’s looking for the wrong fingerprint? That question, whether detection trained on one model carries over to another, is what researchers call cross model AI detection transferability, and the honest answer is “partly, and unpredictably.”

Key takeaways

• A detector trained mostly on GPT-4 text *can* flag Claude and Gemini, but its accuracy usually drops on models it never saw during training. Researchers call that the out-of-distribution problem.
• Detectors learn the statistical fingerprint of a model family, not some universal “this is a robot” signal. Change the model, the decoding settings, or the training data, and the fingerprint shifts.
• Newer models share enough writing habits that transfer is partial rather than zero. That’s why a Claude draft can still trip a GPT-tuned detector.
• A cross-model score is noisier than an in-distribution one. False positives climb when a writer’s natural style is unusual.
• Practical version: don’t assume a clean score on one tool means you’re safe everywhere.

What “trained on GPT-4” actually means

Most modern AI detectors are trained classifiers. You feed them thousands of examples labeled “human” and “machine,” and they learn to tell the two apart. If the machine examples come mostly from GPT-4, the model becomes very good at recognizing *how GPT-4 writes*: its rhythm, its favorite transitions, its preference for tidy, evenly weighted sentences.

Here’s the catch. The detector doesn’t learn a Platonic ideal of “AI-ness.” It learns the specific statistical signature in its training data. Things like token probability patterns, sentence-length distribution, and how often the text takes a low-probability turn. (If those terms are new to you, our explainer on perplexity and burstiness walks through the underlying math in plain English.)

So the real question isn’t “is this text AI?” It’s “does this text look like the AI I studied?” When the answer drifts, accuracy drifts with it.

Why detection transfers at all

You’d think Claude, Gemini, and GPT-4 would be wildly different under the hood. In practice they rhyme more than they clash, and that’s good news for a detector.

Three reasons they overlap:

• Shared training data. They’ve all ingested huge, overlapping slices of the public web. Similar inputs nudge them toward similar phrasings and similar distributions of word choice.
• The same core objective. Each one is trained to predict the next likely token. That tends to produce smooth, fluent, *low-perplexity* prose: text that rarely surprises a language model. Human writing surprises more often.
• Similar polish. Instruction-tuned chatbots all get nudged toward being helpful, balanced, and a little formal. That sands down the quirks a detector would otherwise have nothing to grab.

Because of this shared DNA, a signal learned on GPT-4 doesn’t fall to zero on Claude or Gemini. It degrades, but it carries. Think of it like recognizing a regional accent: if you trained your ear on one city, you’ll still catch a neighboring town’s speech, just with more mistakes.

Why it breaks down

Now the other side. The reason transfer is *partial* and not *complete* comes down to in-distribution versus out-of-distribution text.

In-distribution means the detector is judging text from the same model (and roughly the same settings) it trained on. That’s its home turf, and it’s where published accuracy numbers usually come from.

Out-of-distribution means the text comes from somewhere the detector didn’t see: a different model, a much higher “temperature” setting that makes output more random, a different language, or a heavily edited draft. Performance on out-of-distribution text is almost always worse, sometimes dramatically.

A few concrete things that move text out of distribution:

1. A different model family. Gemini’s phrasing habits aren’t identical to GPT-4’s. A detector keyed to GPT-4’s patterns has a blurrier read on Gemini.
2. Decoding settings. The same model at a low temperature reads more “AI.” Crank the randomness and it scatters the very patterns the detector relies on.
3. Model version churn. “GPT-4” in 2024 and a 2026 release behave differently. Detectors trained on the old one are quietly facing out-of-distribution text the day the new one ships.
4. Human editing. Once a person rewrites the hot spots, the text stops being a clean sample of any model.

This is also why detection has a built-in lag. Every new model launch briefly resets the game until detectors retrain on it.

A made-up scenario: three drafts, one detector

The numbers below are invented to illustrate the pattern, not measured results. Picture a content team running a GPT-4-tuned checker on three product descriptions.

• Draft A came straight from GPT-4 at default settings. The detector nails it and calls it strongly AI. In-distribution, easy call.
• Draft B came from Claude, lightly edited for the brand’s voice. The detector still flags it, but with much less confidence. The softer score reflects partial transfer plus the noise that editing introduced.
• Draft C came from Gemini at a higher temperature, then a human reworked the opening and broke up the long paragraphs. The detector lands below its threshold and calls it human. A miss, but an understandable one. The text drifted far out of distribution.

Notice the pattern. The detector wasn’t *wrong* about AI versus human in some absolute sense. It was confident where the text matched what it knew and fuzzy where it didn’t. That fuzziness is exactly what cross-model transfer looks like in the wild.

Why this matters for writers and teams

The cross-model story has a sharp practical edge, and it cuts both ways.

If you’re a student or writer worried about false flags, understand that an out-of-distribution score is *noisier*, which means more false positives, not just more misses. A 2023 Stanford study (Liang et al.) found GPT detectors misclassified more than half of a set of non-native English TOEFL essays as AI-generated, partly because unusual-but-human style sits in a statistical zone detectors handle badly. When you stack “unusual human style” on top of “a model the detector didn’t train on,” the uncertainty compounds. A score is a probability, not a verdict, and a cross-model score is an especially soft one.

If you’re tempted to game the system by switching models, the same uncertainty works against you. Yes, hopping from ChatGPT to Claude *might* dodge one detector. But the detectors most schools and platforms actually deploy are trained on many model families precisely to close that gap. You might slip past tool A and get caught by tool B, with no way to know in advance which one you’re facing. Model-hopping is a coin flip dressed up as a strategy.

It’s worth remembering why a pure single-model detector barely exists anymore. OpenAI retired its own AI-writing classifier in July 2023, citing low accuracy. The lesson the field took from that era was to stop betting on one narrow signal. The durable move, for both honest writers and the teams checking them, is the same: aim for genuine human variation, not a clean reading on one tool.

That means writing (or editing) with real burstiness, concrete detail, and the small irregularities people actually produce, rather than chasing a number. Running a draft through a checker before you revise is a good way to *see* which sentences read as machine-like, but treat that feedback as a starting point for editing, not a final grade.

Frequently asked questions

Can a detector trained only on GPT-4 reliably catch Claude or Gemini text?

It catches a lot of it, but not reliably across the board. When a detector sees text from a model it wasn’t trained on, accuracy usually drops because the statistical patterns differ. Modern detectors deliberately mix many models into training to shrink that gap, so a pure GPT-4 detector is more thought experiment than shipping product.

Why does detection transfer at all if the models are different?

Large language models share more than they differ. Overlapping training data, the same next-token objective, and similar instruction-tuned polish all push them toward comparable, low-perplexity prose. Those shared habits are what a detector grabs onto, so a signal learned on one model partially carries to another.

Does switching from ChatGPT to Claude fool a detector?

Not dependably. You might dodge one and trip another. Deployed detectors are usually trained on multiple model families, so swapping is a weak play. Real human editing for variety and specificity holds up far better.

How do I know if a tool is testing my draft in- or out-of-distribution?

You usually can’t, because vendors don’t publish their training mix. That uncertainty is the point. Treat any single score as one noisy reading, run more than one checker, and read the result as a probability.

Will cross-model detection get better over time?

Probably, but it’s a moving target. Each new model launch briefly serves detectors out-of-distribution text until they retrain. It’s continuous catch-up, which is one more reason no detector should be the final word on a person’s work.

The bottom line

Cross-model transfer is real but leaky. A GPT-4-tuned detector can flag Claude and Gemini because today’s models share so many habits, yet it does so with less confidence and more error than on its home turf. For anyone writing in good faith, the honest path doesn’t run through guessing which model a checker studied. It runs through writing that genuinely varies and sounds like you. Want to test a draft and edit the parts that read as machine-made? Try PaperBleach and see exactly where your text stands. You can also browse the rest of our writing on how detection works, or see what a plan includes.