Large-scale CelebFaces Attributes (CelebA) dataset
Released: December 2015
The Large-scale CelebFaces Attributes (CelebA) Dataset is a facial dataset developed by a team of researchers at the Chinese University of Hong Kong to help train and test computer vision applications such as facial analysis, facial recognition, and facial detection.
Released late 2015, the dataset consists of 202,599 images of over 10,000 mostly western celebrities, each annotated with 40 attributes such as moustache, beard, spectacles, and the shape of face and nose.
CelebA became a commonly used dataset and is seen to have helped make facial recognition and analysis tools more accurate. It has been referred to and cited in hundreds of academic studies and tests.
However, CelebA has been found to be flawed in important ways.
Accuracy: A University of Nevada research study estimates that at least one third of CelebA images are incorrectly labelled one or more times, making reliable predictions impossible and leading them to conclude that it is 'flawed as a facial analysis tool and may not be suitable as a generic evaluation benchmark for imbalanced classification'. Furthermore, attributes such as attractiveness are highly subjective and subject to cultural and other preconceptions.
Bias/discrimination: CelebA reinforces stereotypes, for instance by labelling Asians with 'narrow eyes' and Blacks with 'thick lips'. The dataset is also comprised of nearly 90% white faces, resulting (pdf) in uneven results in terms of gender, age, ethnicity and other sensitive attributes.
Dual-use: Concerns have been raised that datasets such as CelebA can easily be used to identify, monitor and track people, including minorities and protected groups, unethically and illegally.
Privacy: Little is known about how data was collected to construct CelebA, or whether those whose pictures were used were informed or their consent given.
Safety: CelebA contains potentially insulting labels such as 'fat' and 'double chin', which some people may find insulting or offensive.
News, commentary, analysis
MIT Technology Review; December 10, 2020 - How our data encodes systemic racism
Biometric Update; Nov 27, 2020 - Report says lack of diversity in face biometrics datasets extends to expression, emotion
VentureBeat; July 24, 2020 - Researchers find evidence of bias in facial expression data sets
NBC News; March 19, 2019 - Facial recognition's 'dirty little secret': Millions of online photos scraped without consent
Published: January 2023