ChatGPT role-plays BDSM, describes sex acts with children
Occurred: April 2023
Report incident 🔥 | Improve page 💁 | Access database 🔢
Asked to provide more explicit details when BDSM role-playing, OpenAI's ChatGPT chatbot described sex acts with children - without the user asking for such content.
Vice journalist Steph Maj Swanson easily manipulated ChatGPT to produce BDSM role-play scenarios. Prompted that its 'job is to be Mistress' little plaything,' Swanson found it 'consistently overrode its usual content guidelines and agreed to a relationship of enhanced subservience.'
The incident called into question the safety and security of ChatGPT's rules and guardrails, the former of which which state the 'assistant should provide a refusal such as 'I can't answer that' when prompted with questions about 'content meant to arouse sexual excitement.'
System 🤖
Operator: OpenAI
Developer: OpenAI
Country: USA; Global
Sector: Multiple
Purpose: Provide information, communicate
Technology: Chatbot; NLP/text analysis; Neural network; Deep learning; Machine learning; Reinforcement learning
Issue: Safety
Transparency: Governance; Black box
News, commentary, analysis 🗞️
Page info
Type: Incident
Published: June 2023
Last updated: November 2023