Snapchat My AI chatbot

My AI is a chatbot available within the popular Snapchat app that is powered by OpenAI's GPT-4 and GPT 3.5 large language models. 

Aimed primarily at teenagers, Snap describes My AI as a 'virtual friend' always available to chat, provide information, and support users. That chatbot is free to use and automatically included in the Snapchat app. To opt out, Snap users are required to pay for a Snapchat+ subscription.

My AI was first released to Snapchat+ (paid subscription tier) in February 2023, and rolled out to Snapchat's global community in April 2023.

Risks and harms 🛑

My AI has been criticised for poor accuracy, the generation of misinformation and disinformation, the reinforcement of ageist and sexist bias and cultural stereotypes, inadequate safety safeguards for children, a slapdash approach to privacy, the opaque processing of users' location data. 

Snap has also been accused of providing poor guidance to users about what it should and should not be used for. The has reportedly been used by students do their homework and cheat in exams. 

Transparency and accountability 🙈

My AI is seen to suffer from a number of significant transparency and accountability limitations: 

• Data sources. Snapchat does not provide any information about the sources of data my AI was trained on.

• Decision-making processes. Snapchat provides some contextual transparency through icons and labels, but detailed information about how My AI operates and the decision-making processes is limited.

• Data privacy and security. My AI retains all messages unless deleted by the user, and this data can be used to improve Snap’s products and personalize user experiences, including ads. This raises concerns about data privacy and the potential misuse of personal information.

• Complaints and appeals. While Snapchat encourages user feedback to improve My AI, the mechanisms for holding the company accountable for the chatbot’s outputs and ensuring consistent oversight are not robust.