AI porn engulfs South Korea in "New Nth Room" scandal
AI porn engulfs South Korea in "New Nth Room" scandal
Over 500 South Korean schools and universities have been overwhelmed with deepfake pornography, humiliating and traumatising students and causing outrage amongst the general public.
The deepfakes are reportedly primarily distributed through messaging app Telegram, where anonymous users share pornographic images and video manipulated using AI.Β
The images typically involve superimposing the faces of real individuals onto explicit bodies, causing severe distress, humiliation and long-term psychological effects for the victims.
The Telegram groups - one of which allegedly comprised over 212,000 participants - targeted university students and high school and middle school students.Β
Dedicated "humiliation rooms" were created for some victims.
South Korean offenders are seen to believe they can act without fear of consequences - something activists and commentators attribute to the rapid development of AI technologies and the anonymity and near non-existent content moderation of Telegram.Β
Seoul is also seen to have moved too slowly to regulate deepfake porn, despite previous scandals such as the original Nth Room case and the apparent normalisation of digital sex crimes, particularly among minors who are seen to view creating deepfakes as a form of prank or entertainment.
The revelation sparked public outrage and calls for stricter regulation. Women's rights activists criticised the authorities for not addressing the issue sooner; the country's Advocacy Centre for Online Sexual Abuse Victims reported a sharp increase in the number of underage victims seeking help.
The South Korean government and the country's National Police Agency initiated investigations into Telegram's role in facilitating these activities, following similar actions taken by French authorities against Telegram founder Pavel Durov.
The scandal has been labelled the "New Nth Room", after a 2018-2020 scandal in which sexually exploitative materials involving blackmail, cybersex trafficking and the exploitation of at least 103 confirmed South Korean victims, including children, were distributed through Telegram groups.
β August 2024. South Korean politician Han Jeong Ae introduced a bill aimed at amending existing laws to penalise individuals who purchase, store, or view sexually exploitative content created using deepfake technology.
β September 2024. South Korea's National Assembly passed a law that makes it illegal to view or possess deepfakes that are deemed to be sexually exploitative. People who possess or view an illicit deepfake can be sentenced to up to three years in prison.
Deepfake pornography
Deepfake pornography, or simply fake pornography, is a type of synthetic pornography that is created via altering already-existing photographs or video by applying deepfake technology to the images of the participants.
Source: Wikipedia π
Unknown
Operator:
Developer: Β
Country: South Korea
Sector: Education
Purpose: Generate nude images
Technology: Deepfake - video, image; Machine learning
Issue: Ethics/values; Privacy; Safety
Like many other governments, Seoul has been conspicously slow off the mark to regulate deepfake porn, something largely attributable to its conservative political and civil service culture.
The pressure is not just on Seoul to hold the perpetrators to account; the New Nth Room scandal also puts the onus on Telegram founder Pavel Durov - already under investigation by France and other European authorities - to moderate his platform responsibly and transparently, something he has scrupulously avoided in South Korea and elsewhere.
It will also likely increase pressure on Signal to open its backdoor to governments, given that Telegram devotees are apparently now hot footing it to its rival.
Page info
Type: Incident
Published: September 2024
Last updated: October 2024