Human rights expert questions Ireland's protections in wake of sexualised Grok deepfakes

Vivienne Clarke

The special rapporteur on child protection has questioned if Ireland’s protections holding social media platforms to account are sufficient.

Caoilfhionn Gallagher was speaking on RTÉ radio’s Morning Ireland about concerns over the X platform’s Grok AI tool, which includes the facility to “nudify” images.

"The harms from deep fake sexual abuse for the individuals depicted are equivalent to those from authentic images because, for victims, the videos feel real," said the Irish human rights lawyer. "And given how realistic they are, victims know that they might be perceived as real by others."

Ms Gallagher said that the generation of these images is “often part of a pattern of abuse or harassment", and spoke of the case of Nicole Coco Fox from Clondalkin, who died by suicide due to online abuse.

99 per cent of sexually explicit deepfakes accessible online are estimated to be of women and girls. So this is also a gender based violence issue.

Broader societal issues are also important, she said, referring to other nudification apps aside from Grok.

"We know that they're trained on vast data sets of mostly female images because they tend to work most effectively on women's bodies. And as a result, 99 per cent of sexually explicit deepfakes accessible online are estimated to be of women and girls. So this is also a gender based violence issue."

Ms Gallagher added that there was concern internationally about whether the protections in place were sufficient, because most of the protections from a legal and policy perspective internationally are very focused on the users themselves, who may generate the images, rather than the platforms and the products which facilitate the creation of these images.

“X AI has its own acceptable use policy, which prohibits depicting likenesses of persons in a pornographic manner, but plainly that's completely insufficient.

“Within Ireland, there are existing laws of relevance, particularly in relation to individual users. There's section five of the child trafficking and pornography act 1998. I don't like the term child pornography but that's the term that's used in the legislation. And, of course there's Coco's Law from 2020.

“They're quite focused on the individual users. Now I know there's work being undertaken this week by the Attorney General's office to review the existing framework and there's excellent work being undertaken by Coimisiún na Meán, who are speaking to the gardaí about these issues and also the European Commission.

“But the issue here is Ireland's not alone and internationally there is a concern about the adequacy of the mechanisms for holding the platforms to account.

"Ultimately, this is in part a product safety issue and about whether the product itself, which allows the images to be generated should be illegal or should be regulated more tightly rather than simply the individual users who take action using it.”