Google sentiment analyser thinks being gay is bad
Google sentiment analyser thinks being gay is bad
Occurred: October 2017
Report incident 🔥 | Improve page 💁 | Access database 🔢
A Google sentiment analysis tool exhibited bias against LGBTQ+ identities, raising concerns that it may perpetuate harmful stereotypes and discrimination.
A Vice investigation found that Google Cloud's sentiment analysis system showed evidence of bias against queer terms, assigning more negative sentiment scores to sentences containing LGBTQ+ identities compared to non-queer identities.
This bias could lead to representational and allocational harms for marginalised groups, such as misrepresentation of their views or unfair allocation of resources if the tool is used for triaging or decision-making processes.
The bias likely stems from the tool's training on existing text data, which may contain societal prejudices and negative portrayals of LGBTQ+ individuals.
Google attempted to mitigate this by implementing an "ignore list" for certain identity terms, but this approach is incomplete and can lead to contextual misunderstandings.
For LGBTQ+ individuals, Google's tool may contribute to feelings of marginalisation and stigmatisation, particularly among youth who are already at higher risk of discrimination and mental health issues.
For society at large, it underscores the need for greater awareness of AI biases and the importance of diverse representation in technology development in order to ensure fairer and more equitable treatment of all groups.
Cloud Natural Language API
Operator: Google
Developer: Google
Country: USA
Sector: Multiple
Purpose: Analyse sentiment
Technology: NLP/text analysis
Issue: Bias/discrimination
https://www.vice.com/en/article/j5jmj8/google-artificial-intelligence-bias
https://www.vice.com/en/article/ne3nkb/google-artificial-intelligence-bias-apology
https://www.engadget.com/2017-10-25-googles-sentiment-analysis-api-is-just-as-biased-as-humans.html
https://mashable.com/2017/10/25/google-machine-learning-bias
https://www.techleer.com/articles/387-google-sentiment-analysis-api-gives-a-biased-output/
https://data.blog/2019/08/21/gender-and-racial-bias-in-cloud-nlp-sentiment-apis/
Page info
Type: Issue
Published: February 2025