LAION-400M dataset features racist, derogatory, pornographic content

The LAION-400M dataset contained 'troublesome and explicit images and text pairs of rape, pornography, malign stereotypes, racist and ethnic slurs, and other extremely problematic content,' according to researchers.

A audit (pdf) by University College Dublin, University of Edinburgh and UnifyID researchers found that 'When the researchers typed the word “Korean,” LAION-400M didn’t bring up images of BTS or bulgogi, but naked Korean women. Searching the word “Indian” brought up pictures of South Asian women being raped. 'Best president' brought up images of Donald Trump.'

The researchers argued that LAION-400M and other large datasets are rarely managed in the interests of the individuals and organisations whose data is being collected and used, and that they are unlikely to have been properly cleaned. 

'The rights of the data subject remain unaddressed here' the researchers argue (pdf). 'It is reckless and dangerous to underplay the harms inherent in such large scale datasets and encourage their use in industrial and commercial settings. The responsibility of the licence scheme under which the dataset is provided falls solely on the dataset creator.'

Research, advocacy 🧮

• Birhane A., Prabhu V.U., Kahembwe E. (2021). Multimodal datasets: misogyny, pornography, and malignant stereotypes

News, commentary, analysis 🗞️

• https://www.unite.ai/are-under-curated-hyperscale-ai-datasets-worse-than-the-internet-itself/

• https://www.politico.eu/newsletter/ai-decoded/politico-ai-decoded-us-bill-of-ai-rights-parliament-gets-its-act-together-sort-of-the-dark-side-of-large-ai-models/

• https://info.deeplearning.ai/the-batch-ai-has-a-web-problem-google-goes-multimodal-unfinished-symphony-completed-transformers-get-faster-still-1

• https://www.reddit.com/r/MachineLearning/comments/pmwvw9/p_laion400m_opensource_dataset_of_400_million/