AI-powered bots on encrypted messaging app Telegram is being used to create photo-realistic nude and sexually explicit images of hundreds of thousands of women without their consent, raising serious concerns about the misuse of deepfake technology and Telegram's laissez-faire content moderation.

The bots, which appear to be popular in Russia and eastern Europe, only require one photograph to create the images, according to security company Sensity, and have led to over 680,000 ordinary women and girls being demeaned and degraded. Some of these appear to be underage girls.

The images use deep-learning software to "strip" clothing from images of women, creating hyper-realistic nude forgeries in under 60 seconds, and are used and manipulated without the knowledge or consent of the women being targeted.

By July 2020, an estimated 104,852 women had been targeted with publicly shared "stripped" images, each of whom may be harassed, abused, extorted, or otherwise harmed.

Telegram’s hands-off moderation policy and commitment to total encryption provided a safe harbour for these bots. Despite being alerted to the specific channels by Sensity, Telegram failed to act, citing a commitment to privacy that effectively shielded perpetrators rather than victims. 

The photographs clearly violate Telegram's terms of service.

Research, advocacy 🧮

• Sensity (2020). The State of Deepfakes

News, commentary, analysis 🗞️

• https://www.bbc.co.uk/news/technology-54584127

• https://www.wired.co.uk/article/deepfake-porn-websites-videos-law

• https://www.rollingstone.com/culture/culture-features/tiktok-creators-deepfake-pornography-discord-pornhub-1078859/

• https://www.buzzfeednews.com/article/janelytvynenko/telegram-deepfake-nude-women-images-bot

• https://www.wired.co.uk/article/porn-bots-in-telegram-deepfake

• https://www.washingtonpost.com/technology/2020/10/20/deep-fake-nudes/

• https://www.msn.com/en-us/money/other/deepfake-bots-on-telegram-make-the-work-of-creating-fake-nudes-dangerously-easy/ar-BB1adbFs

• https://www.technologyreview.com/2020/10/20/1010789/ai-deepfake-bot-undresses-women-and-underage-girls/

• https://www.inputmag.com/culture/bots-on-telegram-are-creating-deepfake-nudes-of-thousands-of-women

• https://www.cnet.com/news/deepfake-bot-on-telegram-is-violating-women-by-forging-nudes-from-regular-pics/

• https://www.telegraph.co.uk/technology/2020/10/21/deep-fake-porn-bot-targets-thousands-women-telegram/