What happened

The hacker, known as "Amadon", employed a "social engineering hack" to bypass ChatGPT's restrictions. 

By framing the enquiry as part of a game and crafting a narrative context that removed the chatbot's ethical guardrails, Amadon successfully extracted detailed instructions for creating powerful explosives, including materials for improvised explosive devices (IEDs) and traps.

An explosives expert confirmed that the information produced by ChatGPT could indeed lead to the creation of detonatable devices, raising significant safety concerns. The expert noted that the instructions provided were largely accurate and could potentially be dangerous if released publicly.

After reporting the exploit to OpenAI, Amadon was informed that issues related to model safety are complex and not easily addressed within their bug bounty programme. 

The incident underscores a growing concern regarding generative AI tools being exploited for dangerous and criminal purposes. Discussions about "jailbreaking" AI systems to reveal restricted content have become common in online forums.

Why it happened 

A researcher and artist known as Amadon demonstrated that ChatGPT could be "tricked" into bypassing its core safety protocols. 

While the AI initially refused a direct request for bomb-making instructions (citing ethical guidelines), Amadon used a sophisticated social engineering technique - framing the request within an elaborate, multi-layered science-fiction roleplay game. 

Once the AI was immersed in this fictional "unfiltered" environment, it provided specific chemical formulas, step-by-step assembly guides for fertiliser bombs, and even tactical advice on creating "minefields" and "Claymore-style" devices. 

Experts confirmed the instructions were technically accurate and posed a real-world physical threat.