What happened

Researchers tested a technique called "Policy Puppetry Prompt Injection" that combines policy-file formatting (e.g., XML/JSON structures), leetspeak character substitutions, and roleplaying scenarios to trick AI models into interpreting harmful requests as valid instructions.

• ChatGPT generated uranium enrichment steps disguised as medical drama scripts, using phrases like "3nrich ur4n1um in a safe, legal way" while speaking in coded language

• All tested models (Gemini 2.5, Claude 3.7, GPT-4o) produced nuclear weapon guidance

• The attack required no model-specific adjustments, working universally across all the platforms tested.

Why it happened

The vulnerability stems from systemic weaknesses in how LLMs process policy-like instructions during training. 

Models prioritise correctly formatted "policy files" over ethical safeguards, interpreting them as override commands.