Cork-based research finds deepfake victims struggling to get content removed

Victims of deepfake sexual abuse face major barriers when they attempt to get abusive online content removed and when they attempt to identify the online perpetrators, new Cork-based research has shown.

By analysing public statements from celebrity victims, the researchers found reports of significant psychological and emotional distress as a result of deepfake sexual abuse, as well as social and reputational harm.

The researchers, from the school of applied psychology at University College Cork (UCC), have now called for responses that better protect victims, address the repeated creation and distribution of abusive material, and challenge myths that minimise the harms of deepfake sexual abuse.

The study revealed a dilemma for celebrities who speak publicly about the abuse and how raising awareness can expose them to further harassment and new incidents of deepfake abuse.

Deepfake sexual abuse involves the creation or manipulation of intimate imagery using AI, without consent from the person depicted.

The research found that victims faced claims that deepfake sexual images are not harmful because they are ‘fake’, or that celebrities should expect such content because they are public figures.

The research also found that some victims also described having their imagery placed into online sex work without their consent, with others creating, distributing and, in some cases, profiting from sexual content generated using their likeness.

Victims said reporting online content and seeking its removal was time-consuming and demanding, requiring them to find and document abusive material, often with little success.

Identifying perpetrators was another major obstacle, the study found.Victims also reported being unable to determine who created the deepfakes, with attempts to hold perpetrators accountable often being unsuccessful.

Dr John Twomey, who led the research as part of his PhD studies at UCC, and is now an assistant professor in psychology at the University of Limerick (UL), said speaking out about deepfake sexual abuse comes at a real cost.

“Our study found that celebrities who publicly challenged this abuse or became advocates against it could find themselves targeted with further deepfake content. This creates a troubling situation where victims who speak out to challenge the abuse and protect others may risk exposing themselves to further harm.”