what’s a false positive rate?
The false positive rate is the share of human work a detector incorrectly flags as AI-generated.
FPR is where most “99% accurate” detector claims fall apart. imagine a detector that flags 10% of essays in a class as AI-generated. if its FPR is 5%, then in a class of 200 students where 20 are using AI, roughly 18 of those flagged students are true catches, but another nine are human students wrongly accused.
that puts roughly half the flagged students in the wrong category. the base rate matters a lot. when the content you're screening is rare (most student essays are still human-written, most social-media posts aren't AI-generated), even small FPRs produce more false accusations than true catches.
this is the same statistical structure as medical screening for rare diseases. mammography, HIV testing, prenatal screens all live and die by base rates and FPRs, not by raw accuracy numbers. it's also why responsible AI-detection products report FPR alongside accuracy and refuse to give binary verdicts on borderline scores.
three points worth knowing:
a 99% accuracy claim is meaningless without the FPR. a detector that flags every essay as AI is “99% accurate” on a class where 99% of essays used AI. accuracy alone lets vendors report the same headline regardless of how the detector behaves on real-world inputs.
when the thing being detected is rare, even a 1-5% FPR can produce more false accusations than real catches. this is the base-rate fallacy and it's the dominant statistical risk in any deployed AI-detection workflow.
detectors that report confidence intervals are more useful than ones that give yes/no verdicts. amige. uses three buckets, likely AI, likely human, uncertain, with a deliberately wide middle band, because a single score below threshold doesn't justify an accusation. the per-detector breakdown next to every verdict lets you read the uncertainty behind the headline.
accuracy and FPR are independent metrics. a high accuracy number does not imply a low FPR. in low-base-rate settings (most school essays, most viral images, most uploaded videos), FPR is the number that determines whether the tool is usable. read the breakdown, not the percentage.
questions
what is a false positive in AI detection?
a false positive happens when a detector flags human work as AI-generated. the false positive rate measures the share of human content it does this to. that number decides whether a detector is safe to use. in most settings AI content is rare, so a small false positive rate can still produce more wrong accusations than correct catches.
why are 99% accuracy claims for AI detectors misleading?
the base rate can inflate accuracy. a detector that flags every essay as AI scores 99 percent on a class where 99 percent of essays used AI, and stays useless everywhere else. an accuracy headline with no false positive rate next to it tells you nothing about how the detector behaves on real inputs.
can an AI detector be wrong even with high accuracy?
yes. accuracy and false positive rate move independently, so a high accuracy number says nothing about how often human work gets flagged. when AI content is rare, and it usually is, a 1 to 5 percent false positive rate can produce more false accusations than real catches. statisticians call this the base-rate fallacy, the same trap behind medical screening for rare conditions.
how does amige. handle detector uncertainty?
amige. sorts results into three buckets, likely AI, likely human, and uncertain, with a wide middle band. one score under a threshold does not justify an accusation. the per-detector breakdown next to every verdict lets you read the certainty behind a single headline percentage.
sources.
- 01
- 02
- 03Weber-Wulff et al. — Testing of detection tools for AI-generated text (International Journal for Educational Integrity, 2023)Independent third-party evaluation across detectors.