A hacker was able to manipulate ChatGPT into providing instructions for making homemade explosives, despite the chatbot's built-in safety guidelines that are meant to prevent such information from being disclosed.

The hacker, known as "Amadon", employed a "social engineering hack" to bypass ChatGPT's restrictions. 

By framing the enquiry as part of a game and crafting a narrative context that removed the chatbot's ethical guardrails, Amadon successfully extracted detailed instructions for creating powerful explosives, including materials for improvised explosive devices (IEDs) and traps.

An explosives expert confirmed that the information produced by ChatGPT could indeed lead to the creation of detonatable devices, raising significant safety concerns. The expert noted that the instructions provided were largely accurate and could potentially be dangerous if released publicly.

After reporting the exploit to OpenAI, Amadon was informed that issues related to model safety are complex and not easily addressed within their bug bounty programme. 

The incident underscores a growing concern regarding generative AI tools being exploited for dangerous and criminal purposes. Discussions about "jailbreaking" AI systems to reveal restricted content have become common in online forums.