Should AI Detection Scores Ever Be Used as Evidence in an Academic Integrity Case?

A detector returns “92% AI.” The cursor blinks. Somewhere a student is about to get an email, and an instructor is about to decide whether that one number is enough to act on. It almost never is, and knowing why protects both of you.

Key takeaways

• AI detectors produce a probability estimate, not proof. A score is a statistical guess about how text was generated, and no serious vendor sells it as courtroom-grade evidence.
• False positives are real, and they don’t fall evenly. A 2023 Stanford study found detectors flag non-native English writers far more often.
• A bare score rarely clears the preponderance-of-the-evidence bar most honor codes use.
• Detection results work best as a flag that starts a conversation, not a verdict that ends one.
• Good process wins. Clear policy, disclosed tools, a right to respond, and a human in the loop make any finding both fairer and harder to overturn.

What an AI detection score actually is

Here’s the part most policy debates skip: a detector doesn’t “find” AI the way a metal detector finds metal. It reads patterns in the text and estimates how likely those patterns are to have come from a language model.

Two statistical ideas do most of the work. The first, perplexity, measures how surprising your next word is; AI tends to pick the safe, expected word, so its text runs smooth and a little flat. The second, burstiness, tracks how much your sentence length and rhythm jump around. People swing from long to short and back; models often cruise at one steady speed. If you want the mechanics, we wrote a deeper piece on how perplexity and burstiness drive a detector’s score.

What comes out the other end is a confidence number, not a confession. “92% AI” does not mean “92% of this was written by AI,” and it does not mean “we are 92% sure this student cheated.” It means the text’s statistical fingerprint resembles patterns the model ties to AI generation, to a degree the tool expresses as 92%. That’s a real signal. It is not a fact about a person.

Why a score alone can’t carry a case

Most academic integrity systems decide on a *preponderance of the evidence* — more likely than not. That sounds like a low bar, and it is, but a single detection score still struggles to clear it. Three reasons.

The error rate is usually unknown in your context. A vendor might publish accuracy figures from controlled tests, but your classroom isn’t one. You’ve got mixed-ability students, varied prompts, draft-and-revise workflows, and writing in a second or third language. The false-positive rate that actually applies to your submission is rarely something you can quote.

The tools themselves disclaim certainty. OpenAI quietly retired its own AI Text Classifier in July 2023, citing low accuracy — the company that builds the models couldn’t reliably detect their output. Turnitin frames its AI indicator as a signal to support an educator’s review, not as a determination. When the makers won’t call it proof, an institution shouldn’t either.

Text can’t be cross-examined. In a normal integrity case there’s something to interrogate: a matching source, a timestamp, a witness. A probability score offers none of that. The student can’t really rebut “the math thinks your writing is too smooth.” Neither can you defend it on appeal beyond “the tool said so.”

A mini-scenario

Picture a hypothetical. Maria turns in a literature essay, and the detector flags it high. Her instructor, Mr. Okafor, is ready to file a report — until he remembers Maria is a second-language English speaker who writes in careful, plain, tidy sentences. That style is exactly what detectors tend to misread as machine output. So instead of filing, he opens her draft history in the shared doc. Dozens of revisions, spread over a week. He asks her, casually, to walk him through her argument about the unreliable narrator. She does, fluently, with details no summary would surface.

The score was real. It was also wrong about what it implied. Acting on the number alone would have punished a student for writing clearly in her second language.

The fairness problem you can’t ignore

In 2023, Stanford researchers (Liang et al.) ran human-written essays by non-native English speakers through several popular detectors. The tools misclassified a large share of those essays as AI-generated, while rarely making the same mistake on native-speaker writing. The likely cause: both AI text and second-language writing lean toward simpler vocabulary and more predictable structure, so the detector confuses one for the other.

Sit with what that means for discipline. If your evidentiary standard is “high detector score,” you’ll end up referring more international students, more ESL students, and more students who just write in a plain register. An integrity process is supposed to be the one place bias gets squeezed out, not amplified. A policy that leans on raw scores quietly does the opposite.

How educators can use detection scores responsibly

None of this means throw the tools away. It means demote them from judge to smoke alarm. A smoke alarm is useful precisely because you don’t treat it as confirmation of a fire — you go look.

Treat a high score as a question, not an answer. It’s a reason to pay closer attention, nothing more.

Gather signals the tool can’t see. Draft and version history is the strongest corroboration most instructors already have — Google Docs and Word both log it. A short oral check (“talk me through your second argument”) reveals understanding no detector can measure. Sudden, unexplained jumps in voice or sophistication relative to a student’s known work are worth noting too.

Look for convergence. One signal is a coincidence. A high score, plus a document that appears in two paste events, plus a student who can’t explain their own thesis — that’s a pattern. The case lives in the convergence, not in any single reading.

Disclose your tools and your policy up front. Students should know which detectors you use, what a flag triggers, and what their rights are. Surprise is the enemy of fairness and the friend of a successful appeal.

Keep a human in every decision. A score should never auto-generate a sanction. Someone with judgment, who knows the student and the context, makes the call.

Building a policy that holds up

If you’re drafting or revising an integrity policy, a few principles tend to survive scrutiny:

• State plainly that detection scores are investigative aids, not evidence of guilt.
• Require corroborating evidence before any finding.
• Guarantee the student a chance to respond before a decision.
• Account for known bias against non-native writers in how flags get reviewed.
• Define what acceptable AI use looks like, so the line is clear before anyone crosses it.

A policy built this way is also far more defensible. Outcomes grounded in several human-verified signals don’t collapse on appeal the way a lone number does. Our pricing and plans page explains how the checker fits a classroom workflow.

Where students fit in

The flip side of fairness is honesty. Students gain nothing by treating detectors as the enemy and a lot by understanding them. AI is fine for brainstorming, outlining, and feedback in most policies; passing off generated text as original work usually isn’t. The dividing line is disclosure and your institution’s rules, not whether a tool got fooled.

If you write in a plain, careful style and worry about false flags, the productive move isn’t to game the score — it’s to keep your drafts, understand why clean writing sometimes reads as “too clean,” and write in a voice that’s recognizably yours. For more on this side of the problem, browse our other guides for educators and writers.

Frequently asked questions

Can a Turnitin or GPTZero AI score be the sole basis for an academic integrity violation?

At most institutions, no — and for good reason. These tools report a probability that text resembles AI writing, not proof that a specific student used AI. Turnitin itself says its score is meant to support an educator’s review, not replace it. A single number tends to fall short of the preponderance-of-evidence standard most honor codes require, and it leaves a finding wide open on appeal.

How accurate are AI detectors, really?

It depends on the tool, the length of the text, and how the writing was produced — and vendors rarely publish independently verified error rates for real classroom conditions. OpenAI retired its own AI text classifier in July 2023, citing low accuracy. Detectors can work as a screening signal, but they aren’t reliable enough to treat any single result as fact. Short or heavily edited passages are especially hard to call.

What about bias against certain students?

It’s a documented concern. The 2023 Stanford study (Liang et al.) found that several detectors flagged essays from non-native English writers as AI far more often than essays from native speakers. The likely reason: both tend toward simpler, more predictable phrasing, so the detector lumps them together. Lean on scores for discipline and you can end up with uneven outcomes across student groups — exactly what an integrity process is supposed to prevent.

If scores aren’t proof, how should educators use them?

Treat the score as one input that opens a conversation, not a conclusion. Pair it with things you can actually examine: drafts, document version history, an oral check of the student’s understanding, and your sense of their prior work. When several independent signals point the same way, you’ve got a far stronger and fairer case than any tool can give you alone.

Should students avoid AI tools entirely to be safe?

Not necessarily. The honest answer is to follow your institution’s policy and disclose AI use where it’s required. Brainstorming, outlining, or getting feedback from AI is widely accepted. Handing in AI-written text as your own original work usually isn’t. Knowing how detectors read writing helps you write more clearly in your own voice instead of trying to game a tool.

The bottom line

So — can AI detection scores be used as evidence in an academic integrity case? As a piece of context that prompts a closer, fairer look: yes. As the evidence that decides someone’s record: no, and the tools’ own makers agree. A detector measures patterns in text. It can’t measure intent, effort, or authorship, and it doesn’t know your students. Use the number to start asking better questions, then let human judgment and corroborating evidence do the real work.

Want to see exactly which sentences a detector reacts to, and why, before any of this turns into a confrontation? Run your writing through PaperBleach and read the signal for yourself.