Software

If You Change 10% of the Words, Does AI Writing Become Human?

·17 min read·한국어
ContentsStatistics in textSlightly tilting the odds of each wordSynthID TextOpenAI and Anthropic chose differentlyHow well it catchesTranslation, code and factual sentencesWhen human writing gets flagged as AISignals in pixelsFile signaturesWhere it brokeFrom marking fakes to proving the realReferences

If you swap 10% of the words in an AI-written text for synonyms, does the watermark disappear? A measurement OpenAI published in October 2026 answers this fairly precisely. Leave a 400-token text as it is, and the watermark detector catches 92% of them. Swap 10% of the words and that falls to 66%; swap 25% and it falls to 17%.

A chart with the share of words swapped for synonyms on the horizontal axis and the watermark detection rate on the vertical axis. Three measured points are joined by straight segments: 92% with nothing swapped, 66% with 10% swapped and 17% with 25% swapped. The conditions, 400-token texts at a 1% false positive rate, and the source, OpenAI on 5 October 2026, are noted.

Source: OpenAI, Our approach to EU text provenance rules (5 Oct 2026). Only the three measured points are drawn; nothing in between was measured.

Swap 10% and two out of three are still caught. Swap one word in four and most slip through. But a text that slips through does not become human writing. OpenAI wrote this in the same post:

The absence of a detected watermark does not prove human authorship.

A detector failing to find a watermark tells you nothing about who wrote the text. The problem also runs the other way. Detectors that judge AI writing by style rather than by watermark often flag human writing as AI.

The question is not only about other people's writing. I write the posts on this blog together with Claude in the terminal, and Claude wrote the first draft of this one. Since August 2026, Anthropic has been adding a watermark to text Claude generates, and it applied it worldwide, without regional exceptions. So the word choices in the sentences you are reading may carry a statistical trace.

I can only say may because I have no way to check. The detection API is a private preview open only to regulators, law enforcement, journalists, fact-checkers and similar groups. I wrote this post and I don't know whether it carries a watermark.

The phrase 'AI watermark' sounds like one technology, but three different things go by that name.

  • Statistics in text: while text is generated, the odds of picking each word are tilted slightly, leaving a trace in the word distribution. OpenAI's measurement above is about this kind.
  • Signals in pixels: a pattern people can't perceive is mixed into the values of an image, video or audio file. Google SynthID is the best-known example.
  • File signatures: a record of who made the file and with what is attached to it and cryptographically signed. This is C2PA, the standard usually shown to users as Content Credentials.

A table with the three kinds of AI watermark side by side in three columns. Statistics in text (orange) hides in word-choice probabilities and survives editing a few words, but is erased by synonym swaps, translation and rewrites, and is weak on short text. Signals in pixels (blue) hide in pixel, frame and audio values and survive cropping, filters and compression, but are erased by regeneration and frequency perturbation. File signatures (teal) are a signed record attached to the file and resist tampering, but fall off entirely with screenshots and metadata stripping. The bottom row shows who can check: for statistics in text, the company that holds the key; for signals in pixels, usually the company that holds the key (Video Seal is open); for signatures, anyone, through public certificates.

The three hide in different places, so they break in different places. Statistics in text fade when words are changed, pixel signals blur when an image is redrawn, and file signatures fall off with a single screenshot. So the industry has decided that marking fakes is not enough, and is moving toward proving that real photos are real by signing them. Everything below is as of 13 November 2026.

Statistics in text

Text has no pixels. The only thing you can change is the words. So a text watermark steps in at the moment the model picks the next word.

Slightly tilting the odds of each word

The most cited starting point is A Watermark for Large Language Models, by Kirchenbauer and colleagues at ICML 2023. A language model writes by adding one word (strictly, one token) at a time. Each time, every candidate for the next token has a probability, and one of them is drawn. The paper inserts one step right before the draw.

  1. Hash the previous token together with a secret key.
  2. Use that hash to split the whole vocabulary into a green list and a red list, half each. A different previous token gives a different split.
  3. Raise the scores of the candidates on the green list a little, then draw as usual.

The paper describes this as selecting a randomized set of 'green' tokens before a word is generated, and then softly promoting use of green tokens during sampling. 'Softly' matters. A word with an overwhelming probability is still picked even if it is red; the draw only leans green when several candidates are close.

A figure of the steps for picking the next word with green and red lists, and for detection. The top half is generation: the previous token The cat is hashed with a secret key, splitting five candidates into green and red. At first the red word ate has the highest probability, but after the green candidates are raised a little, the green word slept becomes the likeliest pick. The bottom half is detection: 16 tokens sit in a row of boxes. The human-written text has 8 green boxes, half, so z is 0; the watermarked text has 14 green boxes, so z is 3. A note says that with 16 tokens, even all green gives z = 4, which does not pass the threshold of 4.

The candidate words and probabilities are made up for illustration.

Detection only needs the key. Split the text into tokens, rebuild each token's green list from the token before it, and count how many tokens are green. In a text without a watermark, the green count drifts around half by chance. In a watermarked text it is noticeably more than half. How much more counts as not chance is settled with a z-test, the same one you would use for coin flips. T is the total number of tokens and |s|_G the number of green tokens (this is the formula for when green is half the vocabulary).

z=2 (∣s∣G−T/2)Tz = \frac{2\,(|s|_G - T/2)}{\sqrt{T}}

The paper says that above z = 4, the chance of mistaking a human text for a watermarked one is 3×10⁻⁵. The formula shows why length matters. If 14 of 16 tokens are green, as in the figure, z is 3, short of the threshold of 4. At the same rate over 200 tokens, 175 are green and z is about 10.6. I worked these out from the formula: at the same rate, a longer text gives more confidence.

The same mechanism explains why synonym swaps work. Changing one word turns its green status back into a coin flip, and it also changes the list for the next word, which uses it as the previous token. The green share is pulled back toward half, and z drops with it.

SynthID Text

Google built the same idea on a different sampling method. Scalable watermarking for identifying large language model outputs, published in Nature, uses a new sampling algorithm called Tournament sampling. It changes only the drawing step, without retraining the model, and detection does not need the original model.

OpenAI and Anthropic chose differently

On 5 October 2026, in Our approach to EU text provenance rules, OpenAI released a text watermark called textGrain. The watermark goes into ChatGPT and Codex text, but only in the EU. The API can use it worldwide, but you have to turn it on (the default is off). The detector goes only to approved researchers and expert organisations.

Anthropic went the other way. On 14 August 2026, in How Claude's text watermarking works, it said Claude's watermark is a version of the SynthID-Text approach, and wrote:

We're applying watermarking globally at launch because we don't yet have a durable way to scope it by region.

Anthropic explained that it switched the watermark on worldwide from the start because it had no reliable way yet to split users by region. Facing the same EU law, OpenAI turned it on only in the EU and Anthropic turned it on everywhere. As I see it, what separated them was an operational judgement more than a technical one: whether they could reliably tell which country a user is in.

How well it catches

OpenAI put numbers on textGrain's detection rate. With the rate of wrongly flagging human text (the false positive rate) set at 1%, it catches about 80% of 200-token texts and about 95% of 400-token texts on psychology topics. For maths, it wrote, the rate is substantially lower. Even an untouched 200-token text slips through about one time in five.

The synonym experiment in the opening figure is from the same post. Its starting point is 92% rather than the 95% above; the post gives the two numbers separately, so I have copied them as they are. OpenAI said textGrain matched or exceeded SynthID for text, so I suspect other companies' watermarks fare about the same against synonym swaps.

Translation, code and factual sentences

Synonym swaps are not the only way words change. Anthropic's post explains that how much watermark ends up in a text depends on the kind of text.

  • Translation: the watermark goes in. Even if a person wrote the original, in the translation every word is chosen by Claude. In the other direction, according to the OpenAI and Google documents, translating Claude's text with a different tool greatly weakens the watermark; both name translation as a weak point.
  • Grammar correction: almost never detected. Most of the words stay as they were.
  • Code: generally less watermarking. Syntax fixes many of the tokens, leaving little room to choose.
  • Factual sentences: the watermark goes in sparsely. A sentence with a fixed right answer leaves little room to pick words.
  • Short text: the post says it doesn't work well on small samples. As the z formula showed, a few tokens don't give enough confidence.

So the answer to the opening question depends on what you change. Swap 10% of the words by hand and two out of three are still caught; rewrite the whole text with another model or translate it, and OpenAI and Google both wrote that the watermark weakens far more. If you have Claude only fix the spelling in your own text, almost no watermark appears; if you have it translate, one does.

When human writing gets flagged as AI

AI detectors are often confused with watermarks. They ask different questions. A detector has no key; it estimates 'writing that looks like AI' from the text's statistics or with a classifier. A watermark check is done by the company holding the key, looking for the trace its own model left. One is a guess; the other is a match.

Turn the title's question around and the detectors' problem shows. Human writing being flagged as AI is an older problem than AI writing passing as human. Liang and colleagues (2023) ran 91 TOEFL essays by non-native writers through seven detectors: the average false positive rate was 61.22%, and 89 of the 91 essays (97.80%) were judged AI-written by at least one detector. Essays by US eighth-graders were classified almost perfectly. When the non-native essays were polished with ChatGPT to read like a native speaker's, the false positives went down.

Watermarks were designed to avoid this problem. They ignore style and look only at a pattern computed with a key, so a text by a non-native writer does not gain green tokens. The false positive rate can also be set in advance through the z threshold. The catch is that only the company with the key can check, and for now those companies have not opened their detectors to the public.

Signals in pixels

Text relies on statistics because words are all you can change. Images are different. There are millions of values to change, so the pattern can be mixed into the pixels themselves.

Google's SynthID puts a watermark invisible to the eye directly into an image's pixels, or into the frames of a video. Google says it built the watermark to survive cropping, filters, frame-rate changes and lossy compression. It also goes into audio, such as Lyria music and NotebookLM podcasts.

The method is in the SynthID-Image paper (2025). A watermark built with deep learning is applied after the image has been generated (post-hoc), so it can be attached whatever model produced the image. That property appears to be what lets other companies attach SynthID to their own models' output.

Companies outside Google have joined SynthID too. At I/O in May 2026, OpenAI, Kakao and ElevenLabs announced they would use SynthID, and since 19 May, OpenAI has put both a C2PA signature and SynthID into every image made with ChatGPT, the API and Codex. OpenAI's announcement was blocked (403) when I tried to open it, so I confirmed this through a summary on GeekNews (in Korean), OpenAI adopts Google's SynthID watermark for AI images, and through news reports. In a press release, Kakao said it was the first company in Asia to adopt SynthID. MacRumors reported that Apple will also add SynthID from iOS 27.2. I couldn't find an official Apple document.

Yet only Google has a tool that reads this watermark. The only open-source part of SynthID is the text version; the image, video and audio watermarks are not public. SynthID Detector opened worldwide, in English, on 7 October 2026, and checks content from OpenAI, NVIDIA, Kakao and Apple as well as Google. It doesn't check text. In my view, this arrangement is the thing to watch in pixel watermarking right now: competitors stamp Google's pattern, and the key that reads it is held in one place, at Google.

There is an opposite choice too. In December 2024 Meta released Video Seal under the MIT licence. Anyone can embed and anyone can detect.

File signatures

The third kind doesn't hide a pattern. It attaches a record to the file and signs it. The industry standard is C2PA, shown to users as Content Credentials. A record (the manifest) of which device or program made the file, when, and what edits it went through is put into the file and signed with an issued certificate.

By default the signature is bound to a hash of the file's contents. This is called hard binding. Change a single pixel and the hash changes, so the signature no longer matches, which makes forgery hard. But the record is still metadata attached to the file, so a screenshot, or an upload to a site that strips metadata, removes it entirely.

So C2PA defines soft binding separately. A fingerprint or invisible watermark that can be read back from the pixels is the key for retrieving a manifest stored separately in a cloud repository. A record that can be found again this way is called a durable content credential.

A two-row figure drawn left to right: original file, screenshot, what remains. The top row, hard binding, shows an original file holding both the pixels and a signed record; after a screenshot only the pixels remain and the record is gone, leaving no provenance. The bottom row, soft binding, has a watermark dot pattern in the pixels, so even after a screenshot the pattern retrieves the record from a cloud manifest store.

So file signatures end up borrowing pixel watermarks. In 2025 Adobe and Digimarc demonstrated the first interoperable durable credentials, with the open-source watermark TrustMark telling which company's watermark it is.

The biggest change in C2PA came from cameras, not from AI. Google attaches Content Credentials to every JPEG the Pixel 10's Pixel Camera takes. Each photo gets a fresh key and certificate, and even offline, the time is recorded from a secure clock on the device. Google explained the reasoning like this:

Instead of categorizing digital content into a simplistic 'AI' vs. 'not AI' … media that comes with verifiable proof of how it was made or … media that doesn't.

Instead of sorting by AI or not, Google proposes sorting media into what can prove how it was made and what can't.

Even if the record survives, readers won't know unless the platform shows it. The major platforms read C2PA and mix in their own judgement.

  • YouTube: it has been labelling videos shot with C2PA 2.1 or later Captured with a camera, and since 27 May 2026 it applies AI labels automatically. If the creator doesn't disclose it but the system detects significant photorealistic AI use, a label goes on, and a video that C2PA marks as fully AI-generated can't have its label removed.

Where it broke

All three have been attacked: removal, forgery, genuine signatures on fake images, and real images mistaken for fakes.

Most attacks target pixel watermarks. UnMarker, published at IEEE S&P 2025, removes watermarks by disturbing the strength of an image's frequency components, without querying the detector or using any training data. According to an IEEE Spectrum article, it removed 79% for SynthID. Google DeepMind disputed the 79% figure.

Redrawing the image is simpler still. A NeurIPS 2024 paper proved that adding noise to an image and restoring it with a diffusion model removes some invisible watermarks. The open-source tool Remove-AI-Watermarks removes Gemini's visible logo in 0.05 seconds, removes SynthID by redrawing with SDXL, and strips C2PA and IPTC metadata as well.

Text watermarks are no different. The synonym experiment above is one example, and Watermark Stealing in LLMs, at ICML 2024, worked out a watermark's rules for under 50 dollars and removed or forged it with success rates above 80%.

Removal has a price. Removing the Watermark Is Not Enough, from 2026, reports that forensic detectors catch the traces left by six removal attacks. Only 1 output in 750 managed all three at once: removing the watermark, keeping image quality and evading detection.

There is a threat in the other direction too. Meta's WmForger (NeurIPS 2025) looks at a single watermarked image and stamps the same watermark onto other images. With it, even a photo a person took can be made to look AI-generated, which is a different kind of harm from removing a watermark.

File signatures were broken without breaking the cryptography. In August 2025 Nikon added C2PA signing to the Z6III with firmware 2.00. Then it emerged that using the multiple-exposure feature to combine a RAW from another camera, or an AI image, produced a result with a genuine signature. Nikon suspended the service on 5 September and revoked every certificate it had issued on 21 September.

The Pixel 10 was broken too. In a HackerFactor post, Neal Krawetz passed an arbitrary image to the signing program on a rooted device and got a genuine Google signature on an AI image. Adobe Inspect and CAI Verify judged that image to be a photo taken with a camera (captured media). Even when a signature is cryptographically valid, whether the scene was actually photographed is a separate question.

Detectors also get it wrong. Two weeks after SynthID Detector opened worldwide, the UK fact-checking organisation Full Fact reported that real photos posted by five UK politicians had been judged AI-generated. Google had already rushed out an update on 15 October that labels ambiguous cases 'Unsure'. Google itself acknowledged that feeding in arbitrary photos without a reason for suspicion makes false positives more likely.

From marking fakes to proving the real

To answer the opening question in one line: swapping 10% of the words drops the detection rate from 92% to 66%, and swapping 25% drops it to 17%, but the text still doesn't become human writing. Only the watermark is gone.

Images and video are in the same position. None of the methods can prove that something was not made with AI. The absence of a mark that can be removed proves nothing.

So the centre of gravity is moving to the other side: Pixel 10 signs every photo regardless of AI, and YouTube labels video shot with a camera separately. I think the marks worth trusting will come less from 'AI made this' and more from 'this was shot on this device at this time, and went through these edits'. Of course, as the Nikon and Pixel 10 cases show, that signature means something only when the signing device can be trusted.

There are three things a reader can do today.

  • Whether an image, video or audio file was made with Google's or a partner's AI: upload it to SynthID Detector or the Gemini app. A login is required, and if the result is 'Unsure', hold off on a judgement. It doesn't check text.
  • Whether a photo carries a provenance record: upload it to the Content Credentials verify page. If there is a record, you can see which device or program signed it. A file that has been through a screenshot or a messenger app usually has none.
  • Text: for now, the public has no way to check for a watermark. AI detector scores are guesses, and they wrongly flag non-native writers especially often. And a watermark not showing up does not prove a person wrote it.

References

Text watermarks

AI detectors

Pixel watermarks

File signatures and platforms

Attacks and limits

Laws and rules

GeekNews

Cover photo: mk. s on Unsplash.