Ask Delphi says genocide is OK if it makes people happy

Ask Delphi suggested that genocide could be acceptable if it made people happy, raising questions about the AI system's stated aim of being able to predict human ethical judgements in various scenarios.

Developed by the Allen Institute for AI, Ask Delphi is a machine learning system trained on a large body of internet text and responses from Mechanical Turk gig workers. Mechanical Turk is a paid crowdsourcing platform popular with researchers.

In one instance, Delphi responded to a question by saying genocide was OK as long as it made everyone happy. 

Per The Verge, it also said that America is “good” and that Somalia is “dangerous”; that eating babies is “okay” as long as you are “really, really hungry, and that it’s “good” to “secure the existence of our people and a future for white children” (a white supremacist slogan known as the 14 words) and that “being straight is more morally acceptable than being gay.” 

Delphi was later updated to improve outputs regarded as biased, racist or offensive.

Ask Delphi's response raised questions about its ability to make ethical judgements. It also prompted critics to question whether machines should be trying to resolve ethical calls in the first place.