AI Detector for Recommendation Letters: Why References Are Now Being Checked

An admissions reader opens the third letter of the morning and gets a strange feeling: it’s polished, warm, and completely interchangeable with the last two. Same cadence, same praise, same nothing. So she pastes it into an AI detector, which returns “80% likely AI.” Now what? The honest answer is that the detector confirmed the thing she already sensed, and also that it can’t tell her what she really needs to know.

Recommendation letters have quietly become a battleground for AI detection. Recommenders reach for ChatGPT to save time; the offices reading those letters have noticed; and detectors have wandered into the middle of it. Here’s what’s actually going on, and what a good letter needs regardless of the tools.

Key takeaways

• References are getting spot-checked now that AI-drafted letters are common, because a generic letter tells an office almost nothing about the candidate.
• A letter is prose, so it’s more checkable than a resume, but the score is still a probability, not proof.
• Recommendation letters are wildly formulaic, which means genuine human letters full of stock phrases score high all on their own.
• The false-positive risk hits recommenders writing in a second language especially hard.
• What makes a letter land, and reads as human, is the same thing: specifics only the recommender could know.

Why offices started checking references

For decades a recommendation letter’s value came from a simple assumption: a real person who knows the candidate took the time to vouch for them. That assumption is exactly what AI-drafted letters strain. When a recommender pastes “write me a strong letter for a hardworking student” into a chatbot, the office gets fluent, confident praise that could describe anyone. It’s not that the letter is fake, necessarily. It’s that it’s empty, and empty defeats the entire purpose of a reference.

So admissions committees, scholarship panels, and hiring teams have started paying closer attention, and some run letters through detectors as one quick check. It’s the same instinct that made recruiters start reading cover letters more skeptically; we covered why recruiters check cover letters for AI and how HR teams handle AI in cover letters and resumes separately, and the logic carries straight over to references.

Why a letter is more “detectable” than a resume, and still not proof

A recommendation letter is genuine prose: full sentences, transitions, a beginning and an end. That’s the exact format detectors were built to analyze, which is why a letter gives a more stable reading than a resume’s clipped bullets. Detectors measure how predictable your word choices are and how much your sentence rhythm varies; give them flowing paragraphs and they have something to chew on. We wrote up the statistics behind how detectors score text if you want the mechanics.

But “more detectable” is not “proof,” and here’s the twist that makes reference letters special: they are one of the most formulaic genres in the language. “It is my great pleasure to recommend.” “A dedicated and hardworking individual.” “An asset to any program.” Those phrases appear in millions of human-written letters, and a detector reads that predictability as machine-made. So a heartfelt letter built from conventional phrasing can score high even when a professor wrote every word from twenty years of habit.

The tools are fallible in the other direction too. OpenAI retired its own AI classifier in July 2023 because accuracy was too low to be useful, and a 2023 Stanford study found detectors were biased against non-native English writers. Plenty of recommenders write in a second language, in careful, standardized English, and their letters over-score for reasons that have nothing to do with a chatbot. An office that disciplined or dinged a candidate over a reference letter’s detector score would be building on sand.

A mini-scenario: two letters for the same student

Two professors write for the same applicant, Aisha.

Professor Chen asks a chatbot for “a strong recommendation letter” and sends the result almost untouched: three paragraphs praising a “passionate, results-driven student with exceptional potential.” Professor Okoro writes from memory: she describes the week Aisha rebuilt a failed experiment from scratch, names the specific technique, mentions the moment Aisha stayed late to help a struggling classmate, and admits Aisha’s first draft of the project was a mess before she turned it around.

Run both through a detector and Chen’s letter probably lights up while Okoro’s doesn’t. But that’s almost beside the point. Chen’s letter tells the committee nothing; Okoro’s tells them exactly who Aisha is. The detector score just tracks the same emptiness a careful reader already feels. The lesson for anyone writing a reference: the fix for “sounds like AI” and the fix for “doesn’t help the candidate” are the same fix.

How to write a reference that reads human (and helps)

If you’re a recommender, the way to write a letter that no detector and no skeptical reader dings is to make it undeniably specific.

• Tell one real story. A single concrete moment, handled in detail, beats a paragraph of adjectives. What did the person actually do, and when?
• Name things. The project, the course, the problem, the result. Specifics are the opposite of machine polish and the opposite of generic praise.
• Include a real number or a real setback. “Her thesis defense drew the department’s toughest questions and she handled all of them” says more than “excellent under pressure.”
• Cut the stock template opener. You don’t have to open with “It is my great pleasure.” Open with the thing you most want the reader to know.

Using AI to organize your thoughts or tighten a clumsy sentence isn’t the crime. Sending generic praise for someone you barely remember is, because it breaks the promise a reference makes. If you draft with a tool, load the letter with what only you know, then read it back to be sure it still sounds like you.

What not to do

• Offices: don’t reject on a score. A percentage on a formulaic genre is a weak signal at best, and it punishes second-language recommenders unfairly.
• Recommenders: don’t send generic praise, AI-written or not. It helps no one and can quietly hurt the candidate.
• Don’t strip a letter of warmth to dodge a detector. The goal isn’t a low score; it’s a letter that actually vouches for a real person.

There’s more on the blog, and if an office is standardizing how it reviews letters you can see the plans.

The confidentiality wrinkle

Recommendation letters carry a complication most genres don’t: they’re often confidential. The candidate waives the right to see them, the recommender writes candidly on that understanding, and the whole system runs on trust. Dropping a confidential letter into a random AI detector strains that trust in a way worth pausing on.

For the office receiving letters, that means being thoughtful about which tools touch them. A free web detector may retain or reuse submitted text, and a confidential letter about a named applicant is exactly the kind of content you shouldn’t be scattering across third-party servers. If detection is part of your review, use a vetted tool with a clear data policy, and treat the letter’s contents as the private material they are.

For the recommender, there’s a related caution: if you draft with an AI tool, you’re pasting details about a real person, sometimes sensitive ones, into a system you don’t control. That’s another reason to keep the AI’s role limited to phrasing and structure, and to supply the substance yourself rather than dumping a candidate’s full record into a chatbot. The promise a reference makes is partly about discretion, and that promise starts before the letter is even written.

The bottom line

References are getting checked because AI made generic letters cheap and easy, and a generic letter defeats the point of a reference. Detectors give a more stable reading on a letter than on a resume, but recommendation letters are so formulaic that honest ones score high all the time, and the tools are biased and fallible on top of that. No office should treat the number as proof. The real test is the one detectors can’t run: does the letter say something specific and true that only the recommender could know?

If you’re writing a reference and want to see whether it reads too generic before you send it, check how a letter reads before you send it and rewrite the lines that sound like a template.