Apple Intelligence rewords, prioritises scam messages

Occurred: August 2024

Apple's AI-powered news summarising tool is rewording and prioritising scam messages, making them appear more legitimate to users. 

What happened

Apple's AI-powered feature, part of the "Apple Intelligence" update, has been reported to prioritise and reword scam messages in a way that misleads users into trusting fraudulent communications. 

This has resulted in users receiving scam emails and texts that seem more credible, potentially increasing the risk of falling victim to phishing attacks. 

Users have expressed frustration over the system's failure to effectively filter out these harmful messages, with many noting that important emails may also be deprioritised as a consequence.

Why it happened

The issue appears to stem from the design of Apple Intelligence, which focuses on summarizing and prioritizing messages without adequately distinguishing between legitimate communications and scams. 

This lack of effective spam detection is compounded by the limitations of existing email protocols, which do not support robust server-side filtering, leaving the AI to operate with incomplete information.

What it means

The issue raises concerns about the quality and oversight of Apple's AI service and its readiness for public use.

System 🤖

Operator: 
Developer: Apple
Country: Global
Sector: Media/entertainment/sports/arts
Purpose: Summarise news
Technology: Generative AI; Machine learning
Issue: Accuracy/reliability; Mis/disinformation; Security