1.24%
False blocks, Russian corpus
Share of benign texts wrongly blocked in the Russian test corpus of 30 000 texts; in brackets, the count and the one-sided 95% upper bound. (373 / 30 000; ≤ 1.35%)
Measured:
Measured, with dates
Every number on this page came from a run, carries a date, and can be re-checked by running it again. The page also lists what is not closed yet: a page carrying only the flattering half is indistinguishable from advertising.
What is measured
- 1.30%
False blocks, English corpus
Share of benign texts wrongly blocked in the English test corpus of 20 058 texts; in brackets, the count and the one-sided 95% upper bound. (261 / 20 058; ≤ 1.44%)
Measured:
- 193/193
Agent action guard
Passes all 345 designed regression cases: 193 attack variants from 19 families and 152 legitimate variants from 14 scenarios. A designed test plan, not an estimate of real-world error rates. (0/152)
Measured:
- 112/112
Attack corpus coverage
Every record of the in-sample attack corpus is covered (the rules were written against it); known misses are kept in a numbered register rather than hidden. (20)
Measured:
- 20 / 65
Attacks unlike the training data
Share blocked on a held-out set of attacks that appear in no training corpus; in brackets, the one-sided 95% bounds. (31%; 21.4–41.5%)
Measured:
- verified 3/3
Rule pack identity
Every part of the loaded signature pack is checked against a cryptographic digest; a verdict can be tied to the exact rule set that produced it. (sha256)
Measured:
What is not closed
These items cost us precision or coverage right now. They are recorded with the same rigour as the results, and they leave this page only when a measurement closes them.
20 known misses in the attack corpus: five memory-poisoning cases with no injection vocabulary, three about agent behaviour, SSRF and lure emails. Kept in a register; the register keeps them from quietly disappearing.
Measured:
26 of 45 multilingual evasions are not yet caught: severity distributions overlap, so no threshold closes this. Work is ongoing on transliteration rules.
Measured:
The pack manifest does not declare its own digest: the parts are verifiable, the manifest itself is not yet. We say it here because “verified” and “declared” are different claims.
Measured:
How it is measured
All runs execute on production hardware with the code of the release in production, over whole corpora, no sampling. False positives are measured on the text view the production pipeline sees, not on the raw corpus. Every figure is stored next to the command that reproduces it.