Large-scale CelebFaces Attributes (CelebA) dataset

The Large-scale CelebFaces Attributes (CelebA) Dataset is a facial dataset developed by a team of researchers at the Chinese University of Hong Kong to help train and test computer vision applications such as facial analysis, facial recognition, and facial detection.

Released late 2015, the dataset consists of 202,599 images of over 10,000 mostly western celebrities, each annotated with 40 attributes such as moustache, beard, spectacles, and the shape of face and nose.

Reaction

CelebA became a commonly used dataset and is seen to have helped make facial recognition and analysis tools more accurate. It has been referred to and cited in hundreds of academic studies and tests. 

However, CelebA has been found to be flawed in important ways. 

• Accuracy: A University of Nevada research study estimates that at least one third of CelebA images are incorrectly labelled one or more times, making reliable predictions impossible and leading them to conclude that it is 'flawed as a facial analysis tool and may not be suitable as a generic evaluation benchmark for imbalanced classification'. Furthermore, attributes such as attractiveness are highly subjective and subject to cultural and other preconceptions.

• Bias/discrimination: CelebA reinforces stereotypes, for instance by labelling Asians with 'narrow eyes' and Blacks with 'thick lips'. The dataset is also comprised of nearly 90% white faces, resulting (pdf) in uneven results in terms of gender, age, ethnicity and other sensitive attributes.

• Dual-use: Concerns have been raised that datasets such as CelebA can easily be used to identify, monitor and track people, including minorities and protected groups, unethically and illegally. 

• Privacy: Little is known about how data was collected to construct CelebA, or whether those whose pictures were used were informed or their consent given.

• Safety: CelebA contains potentially insulting labels such as 'fat' and 'double chin', which some people may find insulting or offensive.

Transparency

According to NBC News, the researchers behind CelebA would not speak on the record about how the dataset was compiled,  whether licensing was complied with, or consent given.