Study: OpenAI voice agents can automate phone scams
Occurred: October 2024
Report incident ๐ฅ | Improve page ๐ | Access database ๐ข
OpenAI's realtime voice API can be used to automate phone scams at very low cost, raising questions about the safety of the system, according to researchers.
What happened
Researchers at the University of Illinois Urbana-Champaign demonstrated that OpenAI's realtime voice API can be exploited to automate phone scams at a remarkably low cost - around USD 0.75 per successful scam.ย
The API allows developers to create AI agents capable of impersonating various figures, such as bank officials or government agents, to extract sensitive information from victims.ย
The study revealed that these AI agents could autonomously execute a range of scams, including credential theft and bank account hijacking, with varying success rates.
Why it happened
Recent advancements in OpenAI's technology, specifically the release of its Realtime API, which enables real-time interaction through voice, means anyone can mimic and deloy someone else's voice for any purpose.
Concerns have been raised about the potential for misuse when OpenAI demonstrated voice mimicry without consent, leading to a delay in the release of its advanced voice features.ย
Despite OpenAI's implementation of some safety measures, researchers found that the API's design was simple enough to allow for easy exploitation by malicious actors.ย
The study highlighted that phone scams already affect millions of Americans annually, costing around USD 40 billion, making this new tool particularly concerning.
What it means
The ability to automate phone scams using AI not only increases the scale and efficiency of fraudulent activities but also complicates efforts to combat them. With an average success rate of 36 percent across various scams, and costs as low as USD 0.75 per attempt, the risk to consumers and organisations is significantly heightened.ย
Experts argue that comprehensive solutions are needed at multiple levels -including regulatory measures, enhanced security protocols from phone providers, and stricter oversight from AI companies - to mitigate the growing threat posed by AI-enabled scams.
Intelligent agent
In intelligence and artificial intelligence, an intelligent agent (IA) is an agent that perceives its environment, takes actions autonomously in order to achieve goals, and may improve its performance with learning or acquiring knowledge.
Source: Wikipedia ๐
Operator:ย
Developer: OpenAI
Country: USA
Sector: Multiple
Purpose: Automate voice assistant creation
Technology: Bot/intelligent agent; Machine learning; Speech-to-speech; Voice assistant
Issue: Safety
Research, advocacy ๐งฎ
Fang R., Bowman D., Kang D. Voice-Enabled AI Agents can Perform Common Scams
News, commentary, analysis ๐๏ธ
https://ddkang.substack.com/p/voice-enabled-ai-agents-how-they
https://www.theregister.com/2024/10/24/openai_realtime_api_phone_scam/
https://www.govinfosecurity.com/deepfake-phone-scams-for-less-than-dollar-pop-a-26652
https://protos.com/openai-tool-used-to-create-voice-bot-that-can-drain-crypto-wallets/
Page info
Type: Issue
Published: October 2024