DeepScore trustworthiness assessments accused of bias

Occurred: January 2021

A voice and facial recognition app claiming to score how trustworthy a user prompted privacy and human rights advocates to express arconcerned about the product's accuracy and propensity for bias.

Japanese software company DeepScore uses a 10-question survey to enable loan lenders, insurance companies, and financial institutions to decide in real-time whether people are lying or not with their gestures and tone of voice at a reputed accuracy rate of around 70 percent, and a 30 percent false negative rate. 

Parity AI founder Dr. Rumman Chowdhury told Motherboard the app is at a 'minimum likely to discriminate against people with tics, anxiety, or who are neuroatypical.'

Others are concerned about DeepScore's privacy implications. Ioannis Kouvakas, a legal officer for Privacy International, told Motherboard he did not believe the company would be able to legally operate in the European Union due to the bloc’s General Data Protection Regulation (GDPR).  

DeepScore does not have a privacy policy on its website. It appears that many of its customers are located in countries with threadbare or non-existent privacy laws.

Operator: DeepScore
Developer: DeepScore

Country: Japan

Sector: Banking/financial services

Purpose: Assess user/customer trustworthiness

Technology: Facial recognition; Voice recognition
Issue: Accuracy/reliability; Bias/discrimination - disability; Privacy

Transparency: Black box

Page info
Type: Incident
Published: January 2023
Last updated: September 2023