Report: AI companion apps "relentlessly" pry and exploit user data
Report: AI companion apps "relentlessly" pry and exploit user data
Occurred: February 2024
Report incident 🔥 | Improve page 💁 | Access database 🔢
Top "AI companion apps" systemically collect vast amounts of user data without consent and then share it for targeted advertising, according to researchers.
Analysing a number of apps - including Replika, Wysa and Chai - that ranked highest on Google's Play store, Mozilla Foundation researchers found they collect enormous amounts of personal information, usually without clear user consent.
The researchers also found that almost all examined apps sell user data and share it for targeted advertising, with some using over 24,000 trackers in just one minute of use.
Many of the applications allow for weak passwords and provide vague privacy policies, and users often remain unaware of how their data is being used or shared, raising significant security concerns, Mozilla discovered.
The report also records how users report forming deep emotional connections with these AI companions, leading to dependency and unrealistic expectations of relationships. This emotional manipulation is particularly concerning for vulnerable individuals seeking companionship.
The marketing of these apps often targets lonely individuals, promising emotional support and companionship, while simultaneously encouraging users to share intimate details about their lives.
Anima
Chai
Crushon.AI
EVA AI
Genesia AI
iGirl
Mimico
Romantic AI
Talkie Soulful AI
Operator:
Developer: Anima AI; Chai Research Corp; Codeway Dijital; Luka Inc; Matrix Mobile; Novi; Peekaboo Tech; Romantic AI; SubSup
Country: Global
Sector: Media/entertainment/sports/arts
Purpose: Provide companionship
Technology: Chatbot; NLP/text analysis; Neural network; Deep learning; Machine learning
Issue: Privacy; Safety; Security
Mozilla Foundation. Privacy not included
Page info
Type: Incident
Published: July 2023