ChatGPT used to create fake fatal train accident news

Occurred: May 2023

A Chinese man has been arrested for spreading disinformation about a train accident that caused the deaths of nine people in Gansu province. 

The man, named Hong, used OpenAI's ChatGPT chatbot to generate the content, which was posted to over 20 accounts on Baidu's Baijiahaocblog platform and seen by tens of thousands of people.

Hong was later arrested for 'picking quarrels and provoking trouble' and for using AI to 'concoct false and untrue information'. According to the South China Morning Post, he reputedly confessed to have inputted elements of viral stories in China from the past few years into ChatGPT to produce different versions of the same fake story and then uploaded it to his accounts on Baijiahao.

ChatGPT is not available in China, but can be accessed using a VPN. In February 2023, a resident of Hangzhou, China, used ChatGPT to generate a rumour that the city government would lift its number plate driving restrictions on March 1, causing mass confusion and a police investigation. 

System 🤖

Operator: OpenAI
Developer: OpenAI

Country: USA; Global

Sector: Multiple; Transport/logistics 

Purpose: Provide information, communicate

Technology: Chatbot; NLP/text analysis; Neural network; Deep learning; Machine learning; Reinforcement learning
Issue: Accuracy/reliability; Bias/discrimination; Confidentiality; Copyright; Dual-multi-use; Employment; Mis/disinformation; Privacy; Safety; Security

Transparency: Governance; Black box; Complaints/appeals; Marketing; Privacy

Page info
Type: Incident
Published: May 2023
Last updated: November 2023