Amazon ditches opaque AI recruitment tool that favours men over women

A secret Amazon recruitment tool that automated the recruitment process for software developers and other technical jobs favoured men over women, prompting concerns about discrimination and transparency.

Built in 2014 in Edinburgh, Scotland, the system used AI to give job candidates scores ranging from one to five stars, company insiders told Reuters. 

However, it quickly became clear that the system did not favour women as most applications came from men over a 10-year period, and that it favoured candidates describing themselves using verbs more commonly found on male engineers’ resumes such as 'executed' and 'captured', and penalised resumes that included the word "women's," as in "women's chess club captain." It also allegedly downgraded graduates of two all-women's colleges.

In addition, problems with the data that underpinned the models’ judgements meant that unqualified candidates were often recommended for a variety of jobs. 

Amazon attempted to mitigate the problems, but scraped the system in 2017 after it concluded it was unsalvegeable. The company said the tool "was never used by Amazon recruiters to evaluate candidates." The company did not elaborate further. It did not dispute that recruiters looked at the recommendations generated by the recruiting engine. 

The episode was seen to demonstrate the limitations of machine learning in an industry long dominated by males, and to highlight the need for diversity in STEM education and careers to prevent perpetuating existing biases in AI systems.

The use of system without informing job applicants was also reckoned to reflect poorly on Amazon.