Photo checks for content moderation and trust & safety
Trust and safety teams can add PhotoCheck to an upload queue to label AI-generated images, surface images with faces for privacy review, and read declared provenance like C2PA. One request returns all three, so a moderation rule can combine them without three vendors.
The problem
Platforms now have to label synthetic media, and in some places are legally required to. Self-declared labels miss most of it, and detector scores alone are too noisy to act on automatically. A queue needs both, plus a cheap way to find images with people in them.
The checks that matter
| Declared AI | C2PA and IPTC labels are the strongest signal; act on them automatically. |
|---|---|
| AI score | Rank undeclared images for human review, highest score first. |
| Faces | Route images of people to stricter review. |
A decision rule
Built on POST /api/v1/analyze. Thresholds are starting points; tune them to what a mistake costs you.
const r = await check(upload);
if (r.summary.ai_generated === "declared") label(upload, "AI-generated");
else if (r.ai.score >= 0.8) queue("review:synthetic", r.ai.score);
if (r.faces.count > 0) tag(upload, "contains_people");Questions
Can we auto-label based on the score?
We'd only auto-label on declared provenance. Scores are probabilities; use them to order a review queue.
What throughput can a key handle?
Up to 120 requests a minute per key by default. Write to support if you need more.