LOCAL TEST SHEET / 01

How AI text watermarking leaves a signal

Generate. Inspect. Edit. Check again.

PART A

Run the test

WEBGPU CHECKING

OUTPUT

Generated text

N/A tokensN/A tok/s2.0 bias

Generated text will appear after a run.

One token decision

TOKEN N/A

Waiting for a generated token

WAITING

G+ means that the key placed a candidate in the preferred group. R means regular.

DETECTION

Detection result

WAITING

Eligible positions: N/A
Preferred tokens: N/A
Expected by chance: N/A
z-score: N/A
p-value: N/A

Generate text to calculate a result. No example score is shown.

EDIT AND RECHECK

Change the generated text

0 words

The score updates shortly after you stop typing.

N/Az-scorethreshold 4.00

WAITING

Generate a passage before editing and checking it.

Length retainedN/A

PART B

What happens

An LLM writes one token at a time. A token can be a word, part of a word, or punctuation.

01
Read what came before
The model reads the prompt and the tokens it has already written. It uses this context to predict one new token.
02
Score the next options
The model gives every token in its vocabulary a raw score called a logit. These scores become probabilities. A higher score means a token is more likely, not certain.
03
Choose one token
This demo keeps a set of plausible options, then samples one. Strong options are more likely to be picked. The chosen token is added to the text, and the cycle starts again.
04
Create a preferred group
After the first four generated tokens, the browser derives a key from the demo key. For each new choice, it combines that key, the previous four token IDs, and each candidate token ID. The same inputs always produce the same preferred group.
05
Change the odds
About 25% of the candidates are preferred. The demo adds delta 2 to their logits. It does not force a token or insert a hidden phrase. It only makes preferred options more likely.
06
Leave a pattern in the output
The output is ordinary text. Across many choices, it should contain more preferred tokens than the 25% expected by chance. One token is not evidence. The pattern across a passage is.
07
Verify the text
The detector splits the text into tokens and walks through them in order. At each scored position, it uses the same key and previous four tokens to rebuild the preferred group. It counts the matches and reports a z-score and p-value.
08
Check an edit again
An edit can change a token and the context used for later groups. The detector repeats the same read-only check on the edited text, so the score can rise or fall.

What must match?

Original LLM: No. Verification does not run the model or use its logits.
Tokenizer: Yes. The text must be split into the same token IDs.
Demo key: Yes. A different key creates different preferred groups.
Watermark rules: Yes. Gamma, context width, keyed function, and repeat handling must match.
Delta and sampling: No. Delta, temperature, and top-p affect how strongly the mark is added, but are not needed to verify it.
Decision threshold: Only for the same final label. The z-score can be calculated before a threshold is applied.

This makes verification independent of the original model weights, but not independent of the tokenizer. This app loads the selected model because its tokenizer is packaged with it.

Watermarking is not idempotent. It changes the odds while text is generated; it does not stamp finished text again. Verification is repeatable: the same text, tokenizer, key, and rules produce the same score.

SCOPEThis lab checks for this exact token pattern. It cannot prove who wrote the text, identify a model, or detect all AI writing.

Method and limits

Model: SmolLM2 360M Instruct, q4f16_1
Defaults: gamma = 0.25; delta = 2; context = 4; minimum = 32; threshold z = 4
Result: A z-score and one-sided p-value for this exact setup
Data: The first use downloads the model. Prompts, keys, generated text and checks stay in this browser.
Research basis: A Watermark for Large Language Models

Anthropic says it is working on ways to mark content made by AI, but it has not published Claude's method. This test uses a separate research method.

Read Anthropic's note about Claude

A p-value is the chance of this result or a stronger one under the no-watermark assumption. It is not the chance that AI wrote the text.