Why a Detector Flagging the Bible and the Constitution Should Worry Every Educator

When AI detectors flag famous human texts — the US Constitution, passages of Genesis — as machine-written, they’re revealing exactly how they work: they don’t detect authorship, they detect statistical predictability, and formal human writing is predictable. That’s not a funny glitch. It’s a diagnosis, and it applies directly to the flagged essay sitting in your inbox.

The demonstrations are real and well documented. In July 2023, Ars Technica walked through why pasting the Constitution into GPTZero returned a verdict that the text was likely to have been produced by AI. The opening of Genesis produced similar results on multiple tools. James Madison did not have a subscription to ChatGPT. So what exactly were these tools measuring?

Key takeaways

• Detectors flagged the Constitution and Bible passages as AI because they score predictability (perplexity), not authorship.
• Formal, ceremonial, heavily quoted text is maximally predictable to a language model — and so is your best student’s polished prose.
• The famous-text failures are the same failure that hits ESL writers and disciplined formal writers in your classroom, at scale.
• Vendor tuning has patched some embarrassing demos; it has not changed the underlying method or its false-positive mechanics.
• Use the demo as an AI-literacy lesson, and downgrade every detector score from evidence to, at most, a prompt for a closer look.

Why AI detectors flag famous human texts at all

Strip away the marketing and most AI detectors run on two statistics. Perplexity asks: how surprised is a language model by each next word in this text? Burstiness asks: how much does that predictability vary from sentence to sentence? Low surprise, low variation — the tool leans toward “AI.” High surprise, jagged variation — it leans toward “human.”

Notice what’s absent from that description: any knowledge of who wrote the words. The tool has no access to authorship. It has access to texture. The entire enterprise is an inference from texture to origin, and the famous-text demos show where the inference snaps.

Why the Constitution maxes out the failure mode

The Constitution is close to a worst-case document for this method, for three stacked reasons:

1. It’s formal and ceremonial. Eighteenth-century legal English is rigid, conventional, and rhythmically even — low burstiness by construction.
2. It’s been quoted for two centuries. The text appears throughout every large model’s training data. The model has effectively memorized it, so every next word is maximally unsurprising. “We the People of the United States, in Order to form a more perfect…” — no model on earth is surprised by “Union.”
3. It’s edited to uniformity. Committee drafting sanded off idiosyncrasy, the exact quality detectors read as human.

The same logic covers Genesis, the Gettysburg Address, and most canonical prose. Famous, formal, heavily reproduced writing is statistically indistinguishable from machine output *to a tool that only measures predictability* — because predictability is the one property they genuinely share.

The part that should worry you

Here’s the uncomfortable bridge from a 2023 news story to your classroom this week: your strongest students write like the Constitution. Not in eloquence, necessarily, but in the statistical properties that matter to a detector — controlled syntax, formal register, conventional academic phrasing, evenness that comes from careful revision. Every skill you teach in academic writing pushes prose toward lower perplexity.

The research record confirms the demos aren’t a niche corner case. Stanford researchers found detectors flagged over half of essays by non-native English speakers — writers whose learned, careful English is uniform for the same reason ceremonial English is. Formulaic genres like lab reports misfire the same way. We’ve broken down the real false-positive rate on honest students elsewhere; the Constitution demo is that statistic made visceral.

And the failure isn’t even consistent between tools, which is its own tell. The same essay can score 12% on one detector and 89% on another — we’ve covered why two detectors disagree on the same essay — because each model draws its predictability thresholds differently. A measurement that can’t agree with itself about the Bill of Rights is not a measurement you can discipline a teenager with.

“Haven’t they fixed this by now?”

Partially, and the shape of the fix matters. After the 2023 coverage, vendors tuned their models, and some famous-text demos no longer reproduce on some tools. But look at what that fix is: patching the specific embarrassing inputs, not changing the method. Detection still runs on predictability. The Constitution got an exemption; your ESL sophomore’s essay did not. Meanwhile, OpenAI retired its own AI classifier over low accuracy and never brought it back, and every major vendor’s fine print still says scores aren’t proof of misconduct. Read the fine print as the honest version of the marketing.

What to do with this in your classroom

Turn the failure into a lesson. The demo runs in ten minutes: paste a famous pre-AI text into a detector in front of your students or your department, watch the verdict, then paste a volunteer’s in-class paragraph. You can run any text through a detector and watch the sentence-level heatmap do its judging live — which sentences light up, and why, teaches more about both machine learning and academic writing than a semester of warnings.

Then let the lesson set your policy: scores prompt process checks — drafts, version history, a conversation — and never carry a case alone. There are more walkthroughs on responsible use in the full library of educator guides.

The founders built a system around the idea that accusation requires evidence. It would be a strange irony to suspend that principle based on a tool that thinks they were robots.

Frequently asked questions

Did AI detectors really flag the US Constitution as AI-written?

Yes. In 2023, journalists and researchers showed that pasting the US Constitution into popular detectors — GPTZero among them — returned verdicts that the text was likely AI-generated, and passages from the Bible produced similar results. Ars Technica documented the Constitution case in detail in July 2023. The texts predate computers by centuries, so the flags are pure false positives, and they reveal how these tools actually make decisions.

Why would a detector think the Constitution is AI-generated?

Because detectors don’t detect authorship — they measure statistical predictability, often called perplexity. Text that a language model finds easy to predict scores as machine-like. The Constitution is formal, ceremonial, and quoted so heavily that models have effectively memorized it, making every next word highly predictable. Low perplexity plus uniform rhythm equals an AI verdict, no matter who wrote the words or in which century.

What does this mean for how I treat student detector scores?

It means a high score tells you a text is statistically predictable, not that it came from a machine. Your strongest students — the ones who write polished, formal, disciplined prose — are producing exactly the low-perplexity texture that got the Constitution flagged. A score can prompt a closer look at drafts and process, but if a tool can misfire on the most famous human documents in existence, it cannot be treated as evidence about a sophomore’s essay.

Are newer detectors fixed so this doesn’t happen anymore?

Vendors have tuned models since 2023, and some famous-text demos no longer reproduce on some tools. But the fix is patchwork: the underlying method still scores predictability, so the same failure mode persists for any formal, conventional, or well-edited human writing that wasn’t specifically patched. Tuning out embarrassing headlines is not the same as solving the false-positive problem, and vendors’ own disclaimers still say scores aren’t proof.

Should I show the Constitution demo to students and colleagues?

It’s one of the best ten-minute AI-literacy lessons available. Run a famous pre-AI text through a detector live, watch the verdict, then run a student volunteer’s in-class writing. The demo teaches what detectors measure, why formal writing gets flagged, and why scores deserve skepticism — and it does so more memorably than any policy memo. Just pair it with the point that misuse of AI is still a real issue; the lesson is about evidence quality, not permission.