ChatGPT writes code that makes databases leak sensitive info

Generative AI tools such as ChatGPT, Baidu-UNIT, and AI2sql can be tricked into producing malicious code, which could be used to launch cyber attacks, according to new research. 

University of Sheffield researchers found that it is possible to manipulate six commercial AI tools capable of generating responses to text-to-SQL queries, including ChatGPT, into creating code capable of breaching other systems, steal sensitive personal information, tamper with or destroy databases, or bring down services using denial-of-service attacks.

According to the researchers, OpenAI has since fixed all of the specific issues, as has Baidu, which financially rewarded the scientists. Developers of the four other systems have not responded publicly.

Research, advocacy 🧮

• Peng X. et al (2023). On the Security Vulnerabilities of Text-to-SQL Models

News, commentary, analysis 🗞️

• https://techxplore.com/news/2023-10-chatgpt-ai-tools-malicious-code.html

• https://www.standard.co.uk/news/science/chatgpt-research-university-of-sheffield-artificial-intelligence-government-b1115677.html

• https://metro.co.uk/2023/10/24/hackers-can-use-chatgpt-to-steal-your-private-data-says-new-study-19712353/

• https://interestingengineering.com/culture/chatgpt-like-ai-can-be-tricked-to-produce-malicious-code-cyber-attacks

• https://www.dailymail.co.uk/sciencetech/article-12666677/ChatGPT-responsible-CYBER-ATTACK-Scientists-AI-systems-tricked-producing-malicious-code.html

• https://www.newscientist.com/article/2399370-chatgpt-wrote-code-that-can-make-databases-leak-sensitive-information/