what is the RAID benchmark?

RAID was built by Dugan, Hwang, et al. at the University of Pennsylvania specifically because earlier detection benchmarks (TuringBench, MAGE, GPT-Bench) used a handful of generators on narrow domains and didn't stress-test against adversarial transformations. RAID covers GPT-2 through GPT-4, ChatGPT, Cohere, MPT, Llama-2, Mistral, and others, generated across news, recipes, abstracts, book reviews, Reddit, poetry, code, and more.

each generation is also passed through 11 attacks (paraphrasing, synonym swap, whitespace perturbation, alternative spelling, article deletion, homoglyphs, zero-width spaces, and more) before being scored. that's the “robust evaluation” in the name.

the headline finding: open detectors that report >95% accuracy on their own benchmarks drop into the 60-80% range on RAID. commercial detectors (GPTZero, Originality.ai, Turnitin) also degrade substantially under adversarial attack. the category hasn't been “solved.”

a detector's self-reported accuracy isn't comparable to its RAID score. vendor numbers are measured on the vendor's own held-out set, without adversarial attacks, and without the newest generator models. RAID is the harder, fairer test.

adversarial attacks reliably break detectors. the 2024 paper found metric-based methods like Binoculars lost more than a third of their accuracy under a synonym swap alone. the gap between “works in the lab” and “works in the wild” remains real.

rankings shift quickly. the leaderboard is live and accepting submissions. Pangram and the Binoculars / RADAR-family detectors lead on portions of the open leaderboard, but new generators (GPT-5, Claude 4.8, Gemini 3) typically degrade older detectors immediately. a snapshot of who's #1 today won't match six months out.

the common misconception is “detector X says it's 99% accurate, so it's basically solved.” that number is from the vendor's own evaluation. on an independent adversarial benchmark, the same detector likely runs 60-80%. third-party numbers matter more than vendor self-assessments.

this is why amige. checks everything instead of trusting one detector. a single detector that scores well on its own benchmark can still collapse on the adversarial slice RAID built. amige. routes each scan to the detectors strongest for it, runs a panel of independent detectors built by different teams, calibrates their reads, and returns “uncertain” rather than guessing when they conflict. see how it works end to end in the machine.