ElevenLabs AI voice simulator racism, violence
Occurred: January 2023
Prime Voice AI is a text-to-voice generator for content creators and publishers developed by UK AI start-up ElevenLabs and launched in January 2023 as a beta.
On its website, ElevenLabs describes Prime Voice AI as 'The most realistic and versatile AI speech software, ever.'
Prime Voice AI has been praised by some users for the speed, quality, and realism of its output.
However, it has also been shredded for its lack of safeguards and the ease with which it can be made to generate racist, transphobic, homophobic, anti-semitic, violent, and offensive audio content in other people's names.
4chan and Reddit members quickly used Prime Voice AI to make deepfake voices of Emma Watson (reading out Adolf Hitler's Mein Kampf) and Joe Rogan, amongst others, spewing vile rhetoric, including at US House representative Alexandria Ocasio-Cortez.
The tool's ability to produce convincing deepfakes and other forms of disinformation and misinformation has also been highlighted, as has its potential for copyright abuse and its impact on voiceover industry jobs.
ElevenLabs responded by saying it had seen 'an increasing number of voice cloning misuse cases' and asked for 'thoughts and feedback'.
The following day, the company announced it would insist on verifiying user identities, introduce paid subscriptions, and ban accounts it found to be misusing its technology.
Under a section on Ethical AI on its website, the company says that it is 'fully committed [AIAAIC italics added] both to respecting intellectual property rights and to implementing safeguards against potential misuse of our technology.'
News, commentary, analysis
Published: January 2023