Top AI Companies Commit to Ensuring Children's Online Safety Worldwide

Leading artificial intelligence companies, including OpenAI, Microsoft, Google, and Meta, have united to prevent the exploitation of children and the generation of child sexual abuse material (CSAM) using their AI tools. This initiative was spearheaded by Thorn, a child-safety organization, and All Tech Is Human, a non-profit advocating for responsible tech practices.

Thorn emphasizes that the pledges from these AI firms establish a pioneering standard for the industry, marking a significant stride in protecting children from sexual abuse as generative AI technology evolves. The initiative aims to eliminate the creation of sexually explicit content involving children and remove it from social media platforms and search engines. Notably, over 104 million suspected CSAM files were reported in the U.S. in 2023 alone. Without collaborative measures, the rise of generative AI risks exacerbating this issue and overwhelming law enforcement, which already struggles to identify genuine victims.

Recently, Thorn and All Tech Is Human released a paper titled “Safety by Design for Generative AI: Preventing Child Sexual Abuse.” This document outlines essential strategies and recommendations for companies developing AI tools, search engines, social media platforms, and hosting services. It urges these entities to carefully select training data sets, avoiding ones that contain instances of CSAM or adult sexual content, given generative AI's tendency to merge these concepts. Additionally, Thorn calls on social media platforms and search engines to eliminate links to sites that distribute images featuring child nudity, which could lead to the generation of new AI-produced CSAM.

The influx of AI-generated CSAM could complicate the identification of real victims of child sexual abuse by increasing the “haystack problem”—the overwhelming amount of content that law enforcement must sift through.

“This project aims to show that we can take action,” said Rebecca Portnoff, Thorn's vice president of data science. “Our goal is to redirect the trajectory of this technology to mitigate its existing harms.” Some companies have already committed to separating child images, videos, and audio from datasets that include adult content to avoid the blending of these materials. Others have begun adding watermarks to AI-generated content, although this method is not foolproof, as watermarks and metadata can be easily removed.