AI image detectors are useful, but not as accurate as their marketing suggests. On images like the ones they were trained on, good detectors separate real from generated well. On new generators, screenshots, heavily compressed or filtered photos, accuracy drops, sometimes sharply. Read a score as evidence to weigh, and pick thresholds that match the cost of a mistake.
What PhotoCheck uses, and what we measured
PhotoCheck's AI score comes from umm-maybe/AI-image-detector, an openly licensed (CC BY 4.0) image classifier. We chose it over two Apache-2.0 alternatives after testing all three on images with known provenance.
The test was small: 7 Gemini-generated images and 4 stock photographs. This model scored all 11 correctly and kept them well apart, with generated images at 0.60–0.90 and photographs at 0.02–0.09. The alternatives scored 9/11 (including one confident false positive) and 7/11. Eleven images show the model separates cleanly on that kind of input. They don't give an accuracy rate.
Where detectors fail
False positives (real photos scored as AI) cluster around:
- Beauty filters and heavy retouching
- Strong compression and repeated re-saving
- Screenshots, photos of screens, and scans
- Illustrations and flat graphics, which aren't photographs to begin with
Picking thresholds
Decide what a mistake costs. Auto-rejecting a dating profile at 0.5 will wrongly reject real people; sending everything above 0.8 to a reviewer won't. PhotoCheck's buckets are a starting point: 0.8+ likely AI, 0.5–0.8 possibly AI, 0.2–0.5 probably real, under 0.2 no signs.
Combine signals. Declared provenance outranks any score. Camera EXIF is weak evidence for a real photo. A face check tells you whether a profile photo is a usable portrait at all.