In short: AI detectors don't find a hidden label in a text. They estimate how likely it is that a language model produced the words, based on statistical patterns. Older tools measure perplexity (how predictable each word is) and burstiness (how much sentence length and structure vary); newer ones are trained classifiers that learned the overall “fingerprint” of machine text from millions of examples. Watermarks exist but only work for text from the provider that embedded them. Human writing gets flagged most often when it has been polished by an AI tool, when it is very formulaic, or when it was written in a second language. The most reliable defence isn't a trick – it's being able to show how your text came about.
The two families of AI detectors
1. Statistics: perplexity and burstiness
The first generation of detectors asks a language model a simple question about every word: how surprised am I by this? That surprise is called perplexity. Machine text tends to be low-perplexity – the model picks likely words, so another model finds them unsurprising.
The second number is burstiness: how much the text varies. Human writers mix short sentences with long ones, plain stretches with sudden turns. Model output tends to be more even.
The weakness is built in. Famous, much-quoted texts are extremely predictable to a model that has read them thousands of times, so perplexity-based tools have been known to flag passages from the Bible or the US Declaration of Independence as AI. And plain, careful writing is predictable by design.
2. Trained classifiers
Newer detectors – Pangram, the one we tested with, is an example – are neural networks trained on pairs of texts: a human text and an AI text with the same content and tone. They don't rely on one number; they learn the combined pattern of word choice, rhythm and structure that machine text tends to have. In our tests that made a real difference: a hand-written paragraph scored 0% AI with high confidence.
And watermarks?
Some providers can embed an invisible statistical watermark in their models' output – Google's SynthID for Gemini is the best-known example. A watermark only proves something about text from that provider's model, and heavy revision weakens it. Most detectors you can use today do not read watermarks at all; they analyse the words.
The signals in the text
When people say a text “sounds like AI”, the classifier is usually reacting to a cluster of features, not one word:
- Polish everywhere. Every sentence is fluent; nothing is rough. Each word is the most fitting one.
- Ornament on autopilot. Metaphors and “as if” comparisons in every paragraph, often slightly generic.
- Even rhythm. Sentences of similar length and shape, paragraphs that each end on a neat summary.
- Balanced hedging. “It's not just X, it's Y.” Lists of three. A tidy concluding sentence.
- Low specificity. Plenty of words, few concrete details only the writer could know.
None of these is proof on its own – good human writers use every one of them. Detectors weigh the whole pattern.
Why human writing gets flagged
1. An AI polishing pass. This is the big one, and we measured it. We took that same human paragraph that scored 0%, ran a single “make it more literary and expressive” pass over it – and the result scored 100% AI. In most false positives the detector isn't broken: the text really did pass through a model while it was being “improved”. Grammar tools with rewrite features can have the same effect.
2. Writing in a second language. A widely cited 2023 study from Stanford researchers (Liang et al., published in Patterns) found that several popular detectors misclassified more than half of essays written by non-native English speakers as AI-generated. Simpler vocabulary and more predictable phrasing look “low-perplexity”.
3. Formulaic genres. Abstracts, product descriptions, cover letters, legal boilerplate – text that follows a template is predictable by nature.
4. Short texts. The less text, the less evidence, and the more a single phrase can swing the score.
Accuracy is not a solved problem. OpenAI withdrew its own AI text classifier in 2023, citing its low rate of accuracy. A detector score is a probability estimate – it should start a conversation, not end one.
What actually helps
- Don't add “human messiness”. Sprinkling in filler words and deliberate mistakes didn't work in our tests – it is just another pattern. Switching to a different AI model to rewrite didn't help either: four models, four scores of 100%.
- Leave things out instead. What brought flagged human text back to 0% was simplifying: cut the ornamental metaphors, drop the “as if” comparisons, replace the elevated word with the plain one, break long sentences up. The full experiment is in our workshop report on AI detectors.
- Keep your drafts. Version history, notes and earlier drafts show how a text grew – far more convincing than any score.
- Record provenance as you write. Hermes 3000 keeps a record of which passages you typed and where the AI helped, and can turn that into a verifiable provenance record – the opposite approach to guessing from the finished text.
Frequently asked questions
What do AI detectors look for?
A statistical pattern in the words: how predictable each word is (perplexity), how much sentences vary (burstiness), and – in newer, trained detectors – the overall fingerprint of model-written text. They do not find a hidden tag, except for the rare case of a provider's own watermark.
Can AI detectors be wrong?
Yes. False positives are most common for text that was polished by an AI tool, for formulaic writing, for short texts and for people writing in a second language.
Why was my own writing flagged as AI?
Most often because it passed through an AI tool at some point – a rewrite, a “make it more fluent” step, a grammar tool's suggestion. Without that, very plain or very formulaic writing can also score high on older, perplexity-based detectors.
Do AI detectors detect ChatGPT specifically?
Trained classifiers learn a pattern that is shared across models, so they are not limited to one. In our tests, rewriting with four different models produced the same result.
How can I prove I wrote something myself?
With evidence of the process: drafts, version history, notes, and ideally a provenance record created while you write. A low detector score helps, but it is still only an estimate.
Try it out
Hermes 3000 includes an AI check, a one-click “humanize” tool that removes AI polish from your own text, and a provenance record of how your book came about. Start for free – the first 50 standard pages are free.