Artificial intelligence has revolutionised the way we create and consume content online. From making realistic images to making whole videos in minutes, AI tools have opened up an unprecedented level of creative possibilities. But researchers say that the same technology is now being increasingly used to create violent, disturbing, and potentially harmful content that is spreading rapidly across social media platforms like TikTok and Instagram.
A recent study by the Global Network on Extremism and Technology (GNET) has highlighted a worrying trend when AI-generated videos are coupled with cartoon-like imagery of violence. Alessandra Pugnana, Grazia Ludovica Giardini, and Eleonora Ristuccia have studied how generative AI tools are being used to generate huge amounts of sensational content to generate engagement and drive up views. The study found that platforms are increasingly inundated with what they call “AI slop” content- low-quality, repetitive stuff that is created to take advantage of social media algorithms to make money.
So the current AI video-generating tools like VeoE.ai, PixVerse, Sora, Media.io, and Grok have made it possible for creators to make hundreds of videos as fast as possible and at low costs. There is no need for the traditional video production process to be spent on time and money (with heavy production time, effort, and resources), as AI-powered systems can generate whole scenes, animations, and stories in minutes. With this ease of production, we are not only able to overwhelm social media feeds with content at record speed, but the content creators can have it.
AI slop is closely related to another growing online phenomenon that researchers call “brain rot.” While AI slop is about the content itself, brain rot is the psychological and cultural impact of consuming endless streams of fragmented, overstimulating, and often meaningless content. It is often absurd, surreal, ironic, or nonsensical and is designed to keep users occupied and captivated for long periods of time as they read through algorithm-driven feeds. And such constant exposure can lead to reduced attention spans and affect people’s processing of information online.
One of the most disturbing findings in the study was that AI-generated videos featuring colourful fruits and vegetables in very violent scenarios were popular. These videos of carrots, potatoes, tomatoes, and other innocent little things are often accompanied by graphic animation of murder, torture, and dismemberment. Although the videos may appear innocent and silly, the content is deeply disturbing.
Researchers say the bright colours, cute characters, and cartoon aesthetic may reduce the emotional distance between the violence depicted in an image and the viewer and the violence being depicted. Graphic content might be less serious or acceptable to young people than to adults. The cartoon characters’ benign image can mask the severe nature of the violence that is depicted and may reduce the attention of viewers to such scenes over time.
The study also found content that appeared to be influenced by extremist narratives and violent online subcultures. Some videos may have borrowed visual themes and imagery from cartel violence, terrorist propaganda, extreme-right communities, and nihilistic extremist movements, the researchers said. And while these references are often presented as funny, ironic, or absurd, their repeated exposure can normalise harmful ideologies and violent behaviour.
Experts say that the main concern isn’t individual videos but the cumulative effect of repeated exposure. If kids and teens see violent content so much that it’s funny and fun, violence can become normal; it can become funny or even pleasurable to them. As researchers warn, this normalisation process would likely make vulnerable people more open to extremist messages and online communities that glorify violence.
The findings have revived social media platforms' and AI companies' responsibilities. TikTok, Instagram, and other social media platforms need better moderation systems to track violent content produced by AI before it reaches all users. Also, developers of generative AI tools need to ensure they are in place to prevent the creation and distribution of harmful material.
As AI technology develops, it is more difficult to balance innovation with safety. Generative AI presents great possibilities for creativity and communication, but its misuse highlights that much stronger oversight is necessary for content moderation and digital literacy of users. Without any effective intervention, violent AI-generated content could become a growing threat to online safety, especially for the youth who are attempting to navigate the digital world.
