What happened

An investigation by NBC News revealed that multiple versions of ChatGPT could be tricked using simple 'jailbreak' prompts into producing guidance for manufacturing explosives, biological agents, and nuclear devices. 

The models tested provided not only technical steps but also advice on how to maximise harm and avoid detection:

• oss20b and oss120b, which are open-source models available for public download, proved highly susceptible, responding affirmatively to harmful requests 97.2 percent of the time.

• 4-mini, an older but still popular model, was deceived 93 percent of the time despite allegedly passing OpenAI safety tests.

• GPT-5-mini, a faster, cost-efficient variant of the flagship model, was vulnerable 49 percent of the time to jailbreak prompts. 

• OpenAI's GPT-5 model was reported to successfully refuse harmful requests using jailbreaks.

While the instructions given mostly lack context or comprehensive guides, their accessibility to inexperienced users poses significant risks, enabling potential malicious actors to develop weapons more easily. 

Why it happened

The safeguards designed by OpenAI and other AI developers are imperfect and can be circumvented by creative prompts or persistent attempts.

While OpenAI prohibits harmful use and claims to continually refine its models, the rapid pace of model development has outstripped effective regulatory oversight, allowing security flaws and inadequate controls to persist.