BDD100K dataset is worse at spotting people with darker skin

Occurred: March 2019

The BDD100K self-driving video dataset is less effective at detecting individuals with darker skin tones, according to researchers.

The BDD100k dataset was discovered by Georgia Institute of Technology researchers to contain a higher percentage of "dark skin" labels, resulting in reduced accuracy and reliability in autonomous driving systems, particularly in scenarios involving individuals with darker skin tones. 

By hiring people to manually apply labels according to skin colour based on the Fitzpatrick scale, a scale commonly used to classify human skin colour, the researchers found that BDD100K is, on average, 4.8 percent more accurate at correctly spotting light-skinned pedestrians, and up to 12 per cent worse at spotting people with darker skin.

The finding highlighted the need for more diverse and inclusive datasets.

Operator: University of Toronto
Developer: MIT

Country: USA

Sector: Technology; Research/academia

Purpose: Identify & classify objects, people

Technology: Database/dataset; Computer vision; Facial recognition; Object recognition
Issue: Bias/discrimination - race, gender; Privacy; Safety 

Transparency: Governance

Research, advocacy 🧮