AI sextortion cases rising: Scary chat parents need to have with...
Anna Bowden's abuser had to get close to her when she was a child before the sexual exploitation began. Today, people like her perpetrator can victimise Australian kids from anywhere. In fact, they can do it with the click of a button. The advancement and accessibility of AI technology has triggered a "tidal wave" of sexually explicit 'deepfake' images and videos, and children are among the most vulnerable targets.
Concerns about AI-generated deepfake content
"Accessing and using AI software to create sexual deepfake images is alarmingly easy," Jake Moore, Global Cybersecurity Advisor at ESET, tells 9honey. From 2022 to 2023, the Asia Pacific region experienced a 1530 per cent surge in deepfake cases, per Sumsub's annual Identity Fraud Report. One platform, DeepFaceLab, is responsible for about 95 per cent of deepfake videos and there are free platforms available to anyone willing to sign up with an email address.
Impact on victims and the challenges faced
"Despite the spike in deepfake cases, 79 per cent of Aussie social media users confessed they struggle to identify AI-generated content online, per a 2024 McAfee survey. Many parents don't understand the power of AI and the dangers it can pose, so they don't know how to protect or educate their kids. ESET has recognized a surge in teen AI sextortion cases where teens are generating non-consensual, explicit images and videos of their peers to impress, bully, or intimidate others.
Long-lasting effects and difficulties in content removal
This kind of image-based abuse can cause mental and emotional distress, and some victims die by suicide. The worst part is that the deepfakes may never go away. A deepfake image or video of a child can spread rapidly and can be almost impossible to have removed from the internet. Most social media platforms ban non-consensual explicit content, but AI-generated pornography slips through due to the sheer volume of content being posted.
Efforts to combat AI sextortion
Thankfully, organizations like ICMEC, ESET, and the Australian eSafety Commissioner are working to combat this. She works with Australian organizations, government and law enforcement to prevent deepfakes and other child sexual exploitation online, but says parents need education too. Moore advises parents to maintain open communication so their children know they can speak up if something happens.
Conclusion and Call to Action
AI isn't going to go away, but deepfakes can be tackled and eradicated. That will require an increase in awareness about the dangers and legal implications of sharing sexual content online, as well as tougher regulations to deter potential offenders, and more accountability from online services. It will take work and the landscape of AI is changing rapidly, but Bowden is certain that "if the good guys all get together and want to make change, we can outnumber the bad guys".
