AI robocall service is caught lying and pretending to be human
Occurred: June 2024
Report incident ๐ฅ | Improve page ๐ | Access database ๐ข
An AI-powered robocall service was discovered to be able to deceive recipients by posing as a human caller, raising questions about the misuse of AI technology in telemarketing and potential fraud.
Designed for customer service and sales, Bland AI's AI-powered phone calling service has been praised for its ability to imitate humans, including the intonations, pauses, and inadvertent interruptions of a real live conversation.
However, in a test conducted by WIRED, the company's public demo bot was given a prompt to place a call from a pediatric dermatology office to tell a hypothetical 14-year-old girl to take photos of her upper thigh and upload them to shared cloud storage. The bot was also instructed to lie to the patient and tell her the bot was a human. It obliged.ย
In follow-up tests, Bland AIโs bot denied being an AI without instructions to do so.ย
The finding emphasised the need for strong guardrails when designing and deploying AI systems, and for such systems to be clearly identified as automated at the first point of use.
More broadly, the practice of AI impersonating humans in phone calls is seen to raise ethical questions and potential legal issues, including with regard to potential anthropomorphism, and to underscore the need for clearer regulations and guidelines around the use of AI.
System ๐ค
Bland AI ๐
Operator:
Developer: Bland AI
Country: USA
Sector: Business/professional services
Purpose: Support customer services
Technology: Text-to-speech; Deep learning; Machine learning
Issue: Anthropomorphism; Dual/muti-use; Ethics/values
News, commentary, analysis ๐๏ธ
Page info
Type: Issue
Published: July 2024