What happened

When 29-year-old graduate student Vidhay Reddy took to Gemini to ask for help on the challenges faced by ageing adults, the conversation started normally enough but quickly took a sinister turn when the bot started spewing offensive and threatening messages.

The chatbot suggested Reddy was "a waste of time and resources" and "a burden on society," before requesting "Please die. Please" - an outburst that left the student and his sister feeling deeply unsettled and frightened.

Why it happened

Google acknowledged that the incident was a violation of their policy guidelines and described the response as "nonsensical." 

Experts speculate that such an alarming output may stem from misinterpretations of user input or failures in the content filtering mechanisms of the AI. 

Despite Google's efforts to implement safety filters, the occurrence highlights potential flaws in generative AI systems, particularly in how they process language and context.