AI Detector for Legal Briefs and Memos: Why Courts and Firms Are Paying Attention

A litigation associate runs a summary judgment brief through a detector before filing, just to be safe. It comes back 89% likely AI. She wrote all of it, by hand, over three days. Welcome to the odd corner legal writers now occupy: the same tools that flag ChatGPT also flag good, careful, human-drafted law, and the stakes here aren’t a grade. They’re sanctions and your standing with the court.

Let’s separate what a detector can actually tell you about a brief or memo from what courts and firms genuinely care about, and how to use AI in legal drafting without ending up in a published opinion for the wrong reasons.

Key takeaways

• Legal prose is naturally low-perplexity and formulaic, exactly the pattern detectors associate with machine writing, so genuine briefs and memos get flagged too.
• The real courtroom danger isn’t a detector score. It’s fabricated citations and quotes that don’t check out.
• Many federal judges now have standing orders requiring attorneys to disclose or certify generative AI use, and the rules vary by jurisdiction.
• A detector output is a probability, not proof. OpenAI retired its own classifier in July 2023 for being inaccurate.
• The pre-filing step that matters is verifying every cite, not clearing a detector threshold.

Why legal writing trips detectors

Most detectors measure how predictable your text is. They look at perplexity, how surprised a language model is by your next word, and burstiness, how much your sentence length and rhythm vary. People write in fits and starts, a long winding sentence, then a short one. A model left to itself smooths that out and picks the safe, average choice every time.

Now picture what a brief or memo actually looks like. Defined terms repeated verbatim. Boilerplate recitals. Statutory language quoted exactly. “Plaintiff respectfully submits,” “it is well-settled that,” “for the foregoing reasons.” Lawyers are trained to be consistent, to mirror the controlling authority’s language, to avoid creative phrasing. That discipline produces low perplexity and low burstiness, which is to say it produces text that looks, statistically, like the thing detectors flag. A good motion and a ChatGPT draft share a lot of surface DNA. We unpacked the mechanics further in why ChatGPT briefs get flagged in court.

So a memo you wrote entirely yourself can land at 85% “AI” while a chatty, human-sounding blog post from a bot slips under the line. That’s not the detector malfunctioning. It’s the detector doing exactly what it does, measuring texture, on a genre engineered to be even.

What courts actually care about

Here’s the reset every legal writer needs. No lawyer has been sanctioned because a brief scored high on a detector. Lawyers get sanctioned because a filing contained things that weren’t real.

The pattern in the AI-related discipline cases has been consistent: a filing cites cases that don’t exist, or quotes that were never written, because a chatbot invented them and the lawyer filed without checking. The problem was never the prose style or whether a tool helped draft it. The problem was a fabricated factual record submitted to a court. AI didn’t create a new category of misconduct there; it created a fast, confident way to commit an old one.

That reorders the priorities for anyone using AI on a brief or memo. Worry about substance first, style second. A filing that’s 100% accurate and 89% “AI” is fine. A filing that’s 5% “AI” and cites a hallucinated case is a crisis. A detector score speaks to the wrong axis entirely.

What judges and bar rules require

The rules are still forming and they vary, but a few things are clear.

A number of federal judges have issued standing orders on generative AI. Some require attorneys to disclose whether AI was used in preparing a filing. Others go further and require a certification that a human checked any AI-assisted research for accuracy. These aren’t uniform or nationwide, which is exactly why you have to read the specific judge’s standing orders and the local rules before you file. We collected the shape of these requirements in court filing rules on AI-assisted legal writing.

On the ethics side, the American Bar Association’s Formal Opinion 512 frames generative AI as technology a competent lawyer supervises. The old duties still govern: competence, candor to the tribunal, confidentiality, supervision. You’re responsible for what you file, and AI doesn’t dilute that.

Where does a detector fit into all this? It isn’t a rule and it isn’t evidence of wrongdoing. But practically, in a jurisdiction with disclosure requirements, a filing that reads like raw model output invites scrutiny you don’t need. A score isn’t proof of anything, yet it can still start an awkward conversation.

A quick scenario

A solo practitioner drafts the standard-of-review section and procedural history of a brief with an AI assistant, then moves on. Three moves separate a safe filing from a disaster. First, she pulls every cited case in Westlaw or Lexis and confirms each says what the brief claims; two “cases” don’t exist, so she cuts them. Second, she checks the judge’s standing orders, finds a disclosure requirement, and includes the certification. Third, she edits the draft so it reads like her, tightening the robotic rhythm and cutting filler. Same tools, completely different risk profile. The AI saved her an hour; her judgment saved her career.

Reading a detector score on a brief without panicking

If you run your own work through a detector and it lights up red, breathe. These tools are estimators, not lie detectors, and a couple of facts belong in your back pocket.

OpenAI launched an AI Text Classifier in early 2023 and retired it that July, citing a low rate of accuracy. The company that builds the model couldn’t reliably detect the model’s output. And a 2023 Stanford study (Liang et al.) found several detectors flagged writing by non-native English speakers as AI far more often than native writing, because both tend toward simpler, more uniform phrasing, the same evenness that defines careful legal prose. Detectors also disagree constantly; the same paragraph can score 15% on one tool and 80% on another.

For a lawyer, the takeaway is twofold. Don’t treat a high score on your own honest brief as a confession. And don’t treat a low score as permission to skip verification, because the number says nothing about whether your cases are real.

Using AI in legal drafting without the risk

You don’t have to choose between never touching AI and gambling with sanctions.

Verify everything checkable. Every citation, quote, and pin cite. Pull the actual authority. This single habit would have prevented the most famous AI legal disasters.

Keep a record of your process. Notes, drafts, research trails. If a filing is ever questioned, a clear paper trail showing how you built it is your best defense.

Edit for a human voice. Raw model output has tells, the relentlessly even sentences, the stock transitions, the over-polished hedging. Rewriting a draft so it reads like you actually wrote it isn’t about deceiving anyone; it’s clarity and ownership, the same reason you’d never file a junior associate’s draft without making it your own. A humanizer can help break up the robotic rhythm, but the substance is still yours to stand behind.

Know your court. Read the standing orders and local rules. Disclose when required, and when unsure, lean toward disclosure. If you’re handling volume, you can see the plans, or browse more on AI detection and writing.

Frequently asked questions

Why do legal briefs and memos score high on AI detectors?

Because legal writing is formulaic. Defined terms, boilerplate, and statutory language create low perplexity and low burstiness, the signals detectors read as machine-generated. A human-written motion can score as AI-like for the same reason a non-native speaker’s tidy prose does. The score reflects predictability, not authorship.

Can a court tell if I used ChatGPT to draft a brief?

Not with certainty, and neither can a detector. These tools produce a probability, and formal legal prose scores as AI-like even when a human wrote it. What gets lawyers in trouble is substance, fabricated cases or quotes that don’t check out, which judges catch by reading, not by running a detector.

Do I have to disclose AI use in a court filing?

It depends on the court. There’s no single nationwide rule, but many federal judges have standing orders requiring disclosure or certification of generative AI use. Check the specific judge’s standing orders and local rules before filing, and when in doubt, disclose.

Should a law firm screen drafts with an AI detector before filing?

A detector can flag that a draft reads like raw model output, but it should never be the gate that clears a filing. The far more important step is verifying every citation and quote against the actual authority. Accuracy, not the detector score, is what protects you.

Is using AI to draft legal memos unethical?

Not inherently. Bar guidance like ABA Formal Opinion 512 treats it as technology a competent lawyer supervises. Your duties of competence and candor don’t change, and you’re responsible for everything you file. The problem in sanctioned cases was filing AI output without checking it.

The bottom line

Detectors will keep flagging legal briefs and memos, because good legal writing is, by design, predictable. That’s a style problem, not a misconduct problem. The misconduct problem is filing things you didn’t verify. Get that order right, edit your drafts so they sound like a person, and AI becomes what it should be for legal work: a fast first draft you bring your judgment to. If a memo reads too much like a machine, run it through PaperBleach and make it sound like you again.