CivitAI rewards deepfakes of real people

Occurred: November 2023

Report incident πŸ”₯ | Improve page πŸ’ | Access database πŸ”’

Online AI model marketplace CivitAI has introduced a rewards system that encourages users to create deepfakes of real people.Β 

CivitAI's 'bounties' feature encourages its community to develop deepfakes of real people by allowing users to ask the Civitai community to create AI models that generate images of specific styles, compositions, or specific real people. The person developing the 'best' AI model is rewarded with a virtual currency called 'Buzz'.

In addition to celebrities, 404 Media discovered bounties for private people with no significant online presence, alarming privacy advocates, mental health campaigners and others concerned about the potential use of CivitAI to create non-consensual AI-generated sexual images of real people.

According to 404 Media, CivitAI says bounties should not be used to create non-consensual AI pornography.Β 

But the company has been accused of turning a blind eye to multiple instances of deepfake, non-consensual sexual imagery discovered on its platform.

Deepfake

Deepfakes (a portmanteau of 'deep learning' and 'fake') are images, videos, or audio which are edited or generated using artificial intelligence tools, and which may depict real or non-existent people. They are a type of synthetic media.

Source: Wikipedia πŸ”—

System πŸ€–


Documents πŸ“ƒ

Operator: Β 
Developer: CivitAI
Country: USA
Sector: Media/entertainment/sports/arts
Purpose: Generate images
Technology: Text-to-image; Generative adversarial network (GAN); Neural network; Deep learning; Machine learning
Issue: Business model; Ethics/values; Privacy

Page info
Type: Incident
Published: November 2023