Personal images of Australian children have been used to train major AI models, violating their privacy and potentially resulting in the use of their images to create pornographic deepfakes.

According to Human Rights Watch, the photos of 190 Australian children were scraped from personal blogs, video and photo-sharing sites, school websites, photographers’ collections of family portraits and other services without the consent of the children or their parents to create the LAION-5B dataset.

The photos were then used to create popular generative AI tools such as Stable Diffusion and Midjourney and, HRW argues, were later used to create synthetic images that could be categorised as child pornography. 

HRW said it found children whose images were in the dataset were easily identifiable, with some names included in the accompanying caption or the URL where the image was stored. 

The finding raised ethical concerns about data privacy and the need for consent when AI training datasets.