Genderify accused of gender bias
Genderify accused of gender bias
Occurred: July 2020
Report incident 🔥 | Improve page 💁 | Access database 🔢
AI gender identification tool Genderify faced accusations of gender bias before being quickly shut down.
An AI-powered API service launched in July 2020, Genderify claimed to be able to determine a person's gender from their name, email address or username and was was marketed as a tool for businesses to personalise communications and target marketing.
However, Genderify was criticised for producing biased and stereotypical results almost immediately after launch, including misgendering people with non-Western names and demonstrating clear biases, such as associating terms like "doctor" with male and "nurse" with female.
In addition, the company's API returned probabilities for only binary male/female options, ignoring non-binary identities.
Genderify was taken offline after a few days of intense criticism on social media, and the company's website and social media accounts deleted without explanation.
The episode was seen as a cautionary tale about the need for diverse teams and extensive bias testing in the development of AI systems.
Genderify
Operator:
Developer: Genderify
Country: USA
Sector: Technology
Purpose: Identify gender
Technology: NLP/text analysis; Machine learning
Issue: Accuracy/reliability; Bias/discrimination
Page info
Type: Issue
Published: August 2024