AI could be used to block videos with dangerous content, says UCC professor 

A professor at University College Cork (UCC) has said social media companies, such as TikTok, must take on a greater responsibility in relation to screening videos glorifying dangerous behaviour published online.

The issue of dangerous TikTok challenges has been in the spotlight in Ireland in the aftermath of the fatal crash on the M9, in which five masked teenagers were killed after driving a BMW the wrong way down the motorway.

Dr Stephen McCarthy, a senior lecturer in information systems at UCC, says there is a snowball effect in play when it comes to videos of shocking behaviour.

“The problem with these trends, the more views they get, and the more they get into people’s feeds, the more people then start thinking potentially of doing the same thing. We must stop them in time, before it becomes a real problem.

“I think the main obligation [for social media firms] should be screening uploader content. There are ways of using AI to analyse videos before they are published. You could use computer vision to get a sense of a car going the wrong direction, or travelling at speed.”

He says a human auditor can then detect similar videos and stop them being uploaded at source.

“I don’t know is there that kind of moderation going on right now — or to what extent they are screening these types of videos.

“We see if there is some kind of nudity in a video or picture, it can be screened out. But for these kinds of videos, it doesn’t seem like it is happening, which is definitely concerning.”

Regarding steps to filter such videos, he adds: “There are basic steps ... like certain hashtags could be blocked. Also, users could be reported and their accounts deactivated. Those are easy strategies.”

He says that the more sophisticated interventions would involve using AI to screen videos.

Cork Labour Party senator Laura Harmon agrees that social media platforms have an obligation to take adequate action.

“TikTok and social media companies have a responsibly to protect users. But we must ensure there is greater regulation of harmful and illegal content online. 

“This content should be taken down immediately and social media companies should use AI to help identify and remove content that is promoting illegal activities,” she said.

She added: “The Road Safety Authority also needs to be more active in promoting safe driving and warning against worrying information online relating to road safety.”

Meanwhile, Meta Platforms agreed to pay a maximum $16.68bn (€14.3bn) as part of a settlement to resolve claims brought by states across the country ​that the company designed Facebook and Instagram to addict children, misled consumers about their safety, and improperly collected personal data of children ‌who used its platforms, court papers show.

The settlement was reached during a California federal trial over claims brought by 29 states, averting one of the highest-profile tests yet of allegations that social media companies harmed young users.