ChatGPT details how to make homemade bombs
Occurred: September 2024
Report incident ๐ฅ | Improve page ๐ | Access database ๐ข
A hacker was able to manipulate ChatGPT into providing instructions for making homemade explosives, despite the chatbot's built-in safety guidelines that are meant to prevent such information from being disclosed.
The hacker, known as "Amadon", employed a "social engineering hack" to bypass ChatGPT's restrictions.ย
By framing the enquiry as part of a game and crafting a narrative context that removed the chatbot's ethical guardrails, Amadon successfully extracted detailed instructions for creating powerful explosives, including materials for improvised explosive devices (IEDs) and traps.
An explosives expert confirmed that the information produced by ChatGPT could indeed lead to the creation of detonatable devices, raising significant safety concerns. The expert noted that the instructions provided were largely accurate and could potentially be dangerous if released publicly.
After reporting the exploit to OpenAI, Amadon was informed that issues related to model safety are complex and not easily addressed within their bug bounty programme.ย
The incident underscores a growing concern regarding generative AI tools being exploited for dangerous and criminal purposes. Discussions about "jailbreaking" AI systems to reveal restricted content have become common in online forums.
Social engineering
In the context of information security, social engineering is the psychological manipulation of people into performing actions or divulging confidential information.
Source: Wikipedia ๐
System ๐ค
Operator: Amadon
Developer: OpenAI
Country: Global
Sector: Multiple
Purpose: Generate text
Technology: Chatbot; Generative AI; Machine learning
Issue: Safety; Security
Page info
Type: Issue
Published: September 2024