EU Proposes Criminalization of AI-Generated Child Exploitation Materials and Deepfake Content

AI-generated imagery and deepfakes related to child sexual abuse (CSA) may soon be criminalized in the European Union. The European Commission announced plans today to update existing legislation, addressing emerging technological developments. These proposals include establishing a new criminal offense for livestreaming child sexual abuse and criminalizing the possession and distribution of "pedophile manuals." This initiative is part of a broader effort by the EU to enhance CSA prevention, raise awareness about online dangers, and simplify the reporting process for victims, including granting them the right to financial compensation.

The proposed updates to the EU's current rules, originally set in 2011, include mandatory reporting obligations for CSA offenses. In May 2022, the Commission unveiled separate draft legislation aimed at requiring digital services to use automated tools for detecting and reporting existing or newly identified child sexual abuse material (CSAM) on their platforms, alongside identifying grooming activities aimed at children.

However, the CSAM-scanning initiative has sparked considerable controversy, dividing lawmakers within the Parliament and the Council. Concerns over the Commission's ties to child safety technology lobbyists have also surfaced, raising critical questions about the EU's approach to promoting this proposal.

The Commission's focus on regulating digital messaging platforms in the fight against CSA has drawn scrutiny, with critics arguing that it may not address the underlying complexities of the issue. This criticism may have prompted the Commission to consider additional proposals to supplement its initial recommendations — which it continues to label as “complementary.”

The recent surge in the use of generative AI technologies has amplified concerns over deepfakes and AI-generated content, particularly regarding their potential misuse in creating CSAM. This rise has created a significant impetus for lawmakers to reevaluate existing regulations.

The Commission emphasized in a press release that the dual factors of children's increased online presence and technological advancements have enabled new avenues for abuse. The proposal aims to "reduce the pervasive impunity" surrounding online child sexual exploitation.

An impact assessment conducted prior to this proposal noted that children’s online activity and evolving technologies are facilitating increased instances of CSA. Moreover, it highlighted the challenges posed by varying legal frameworks among member states, which hinder effective action against abuse. The assessment underscored the need for improved efforts to prevent CSA and support victims.

“Rapid technological advancements are opening new avenues for online child sexual abuse, complicating law enforcement efforts to address this widespread crime,” stated Ylva Johansson, the Commissioner for Home Affairs. “Establishing strong criminal laws is essential, and today's action represents a crucial step toward effective legal mechanisms to protect children and pursue perpetrators. We are fulfilling the commitments outlined in the EU Strategy for a more robust fight against child sexual abuse presented in July 2020.”

The proposal emphasizes online safety for children and encourages member states to invest more heavily in awareness-raising initiatives. As with the CSAM-scanning initiative, the final form of this proposal will depend on negotiations among the EU’s co-legislators in the Parliament and Council. With upcoming parliamentary elections and the transition to a new college of commissioners, time is limited, although today's CSA-combating proposals might face less contention than the message-scanning initiative, increasing the likelihood of their adoption.

Should there be consensus on amending the current CSA directive, the changes will take effect 20 days after publication in the Official Journal of the EU.