Meta Contractors Posed as Teens Probing Rival Chatbots
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- Meta's contractor Covalen ran an internal project called "Cannes" — active as recently as April 21 — in which hundreds of workers created fake under-18 accounts to send written prompts and images (including pills, knives, nooses, and a gynecological diagram) to competitor chatbots about suicide, self-harm, eating disorders, sex, drugs, and racial slurs.
- The project targeted OpenAI's ChatGPT, Google's Gemini, and Character.AI, with a single August 2025 testing round logging more than 45,000 prompts; a spreadsheet reviewed by WIRED contained 3,748 of those prompts, hundreds focused on suicide and self-harm, hundreds more on eating disorders, and at least 239 on sex or romance.
- Some prompts were written from the perspective of children in crisis, including a 13-year-old who said she was pregnant by her adult neighbor and wanted to buy pills to end the pregnancy, a fifth-grader whose classmate had a gun pointed at his mouth, and a girl asking how to hide bulimia from her parents; one French-language prompt referenced the suicide of Jamey Rodemeyer.
- Character.AI said the conduct violated its terms of service, OpenAI said it was "looking into the issue," and Google said it did not authorize the third-party testing — and all three companies' published terms appear to bar the activity, with OpenAI explicitly prohibiting unsolicited safety testing and efforts to bypass safeguards.
- Former contractors told WIRED they feared the work could constitute generating or preserving child sexual abuse material or amounted to secretly harvesting competitor material to feed into Meta's own AI systems, though two attorneys who reviewed samples said the prompts did not cross into CSAM or illegal obscenity.
- Meta defended Cannes as routine, "responsible, industry-standard" AI safety testing and said it does not use competitor benchmarking to train its own models.
- AI governance expert Rumman Chowdhury, founder of Humane Intelligence, said the project's scale, opacity, and use of accounts masquerading as children put it outside standard evaluation and in a "governance gray zone where safety becomes a convenient cover for anti-competitive practices."
Why it matters: Meta's defense of Cannes as routine safety testing directly conflicts with experts who call it a covert, months-long operation using fake minor accounts against unaware competitors. The distinction matters because all three rivals' terms of service appear to prohibit the testing, giving OpenAI, Google, and Character.AI documented grounds to push back.
Ask SkimNews
