Speech2Face facial reconstructions

Speech2Face is an algorithm developed by a group of researchers at MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) and Google AI that generates images of what someone might look like from snippets of audio recordings of their voice.

According to its associated research paper (pdf), the MIT researchers used a dataset of millions of clips from YouTube and elsewhere and created a neural network-based model that learns vocal attributes associated with facial features from the videos.

Risks and harms 🛑

Concerns have been expressed about Speech2Face's accuracy and reliability, potential for discrimination, and violations of privacy.

• Reaction to Speech2Face has been mixed, with some commentators praising it for creating rough likenesses of people with little information, whilst others focused on the fact that the images created by Speech2Face often only bear a general resemblance to the speaker, and that the system is prone to creating images of the wrong gender and ethnicity. 

• Others highlighted the system tends to identify people with high voices as female and low voices as malem and people speaking Asian languages as Asian, reflecting strong gender, racial and country of origin biases in its data and results. Some went further, accusing it of 'ethnic profiling at scale' and being little more than 'awful transphobic shit'. 

• Speech2Face also kicked off a debate about the nature of data privacy, with some questioning whether this kind of technology could be used to identify individuals, despite the team claiming their method 'cannot recover the true identity of a person from their voice.'

• An individual included in the dataset told Slate that he didn’t remember signing a waiver for the YouTube video he was featured in that was fed through the algorithm. It also prompted a conversation about the need for ethics review boards at conferences and funding agencies.

News, commentary, analysis 🗞️

• https://www.newyorker.com/tech/annals-of-technology/who-should-stop-unethical-ai

• https://hub.packtpub.com/speech2face-a-neural-network-that-imagines-faces-from-hearing-voices-is-it-too-soon-to-worry-about-ethnic-profiling/

• https://gizmodo.com/an-algorithm-generated-eerily-accurate-portraits-based-1835327568

• https://slate.com/technology/2019/06/youtube-twitter-irb-human-subjects-research-social-media-mining.html

• https://www.fastcompany.com/90357561/this-ai-guesses-human-faces-based-only-on-their-voices

• https://www.iflscience.com/technology/new-ai-figure-out-look-like-just-sound-voice/

• https://medium.com/swlh/the-ai-that-knows-your-face-from-your-voice-90772b352f2a

• https://www.sciencealert.com/this-ai-tries-to-guess-what-you-look-like-based-on-your-voice

• https://www.livescience.com/65689-ai-human-voice-face.html

• https://www.diyphotography.net/creepy-ai-reconstructs-your-portrait-based-only-on-your-voice/

• https://futurism.com/the-byte/ai-guesses-appearance-voice