OpenAI details textGrain watermark detection limits

OpenAI says its textGrain watermark detector cannot reliably identify short passages and can lose its signal after substantial rewriting, paraphrasing or translation. That limits what a negative result can establish about AI-generated text. The company is using the watermark for ChatGPT-generated text in the European Union and lets API customers opt in for supported models, but it has not described the system as a global default.

The limitations matter because Article 50 of the EU AI Act requires providers of systems that generate synthetic text to mark outputs in a machine-readable format and make them detectable as artificially generated or manipulated. The methods must be effective, interoperable, robust and reliable as far as technically feasible. Official materials reviewed for this story do not specify a single detection-rate threshold that determines compliance, and they do not establish whether regulators consider textGrain compliant or noncompliant.

textGrain creates an invisible statistical signal by changing how a model selects among plausible words or word pieces. Its detector uses keyed randomness to test whether the resulting token pattern is consistent with OpenAI’s watermark. Short passages contain fewer such choices, while code and tightly constrained factual text can be harder to mark because fewer plausible alternatives are available.

Length and subject matter both affected detection in OpenAI’s evaluations. At a target 1% false-positive rate, the company said its detector identified about 80% of watermarks in 200-token psychology passages and about 95% in 400-token passages. Detection was substantially lower for mathematics content. These are company-reported results, and the evidence reviewed for this story contains no completed independent replication.

Editing caused a steeper decline in another OpenAI evaluation. For 400-token passages, replacing 10% of words with synonyms reduced detection from about 92% to 66%, while replacing 25% cut it to 17%. OpenAI’s separate technical report describes the generation and detection procedures, calibration and theoretical assumptions; the performance figures appeared in the accompanying company post.

OpenAI cautions that neither detector outcome amounts to a complete authorship judgment. A missed watermark does not prove that a person wrote the text because an OpenAI-generated passage may be too short, edited, translated, unsupported or created before watermarking was available. A detected signal means an OpenAI model likely generated or processed the content, but it does not identify the author, measure human contribution, establish accuracy or determine whether disclosure was legally required.

The company has initially limited detector access to approved researchers and expert organizations. OpenAI said the risk of missed watermarks and false positives was why it was not making the detector public at launch. The European Commission describes its transparency code as a voluntary compliance tool, while the underlying Article 50 obligations have applied since August 2, 2026.