Proctorio found to use 'racist' algorithms to detect students' faces

A facial detection system run by online proctoring service Proctorio was found to be 'racist'. 

College student Lucy Satheesan reverse-engineered the exam software and discovered that Proctorio's software had been using a facial detection model that failed to recognise Black faces 57 percent of the time. 

By assessing the code behind Proctorio’s extension for the Chrome web browser, college student Lucy Satheesan discovered that the file names associated with the tool’s 'proprietary' facial detection function were identical to those published by open-source computer vision software library OpenCV.

Tbe discovery appeared to confirm complaints by students that Proctorio's face detection system is inaccurate and unreliable. 

The company uses the technology to see if a student is looking away from their screen, leaves the room, or if there’s another person in the frame - any of which could indicate cheating.

➕ The Electronic Privacy Information Center (EPIC) had earlier filed a complaint (pdf) accusing Proctorio and four other online test proctoring services of unfair and deceptive trade practices. EPIC said it is preparing to file a lawsuit unless they change their practices.

Legal, regulatory 👩🏼‍⚖️

• EPIC (2020). In re Online Test Proctoring Companies

• EPIC (2020). Letter to Proctorio (pdf)

• EPIC (2020). Complaint and Request for Investigation, Injunction, and Other Relief (pdf)

• Senator Richard Blumenthal (2020). Blumenthal Leads Call for Virtual Exam Software Companies to Improve Equity, Accessibility & Privacy for Students Amid Troubling Reports

Research, advocacy 🧮

• Proctor Ninja (2021). Proctorio's facial recognition is racist

• Proctor Ninja (2021). The duality of obfuscation: feat. Proctorio

News, commentary, analysis 🗞️

• https://www.vice.com/en/article/g5gxg3/proctorio-is-using-racist-algorithms-to-detect-faces

• https://www.vice.com/en/article/n7wxvd/students-are-rebelling-against-eye-tracking-exam-surveillance-tools

• https://www.theverge.com/2021/4/8/22374386/proctorio-racial-bias-issues-opencv-facial-detection-schools-tests-remote-learning

• https://www.vox.com/recode/22175021/school-cheating-student-privacy-remote-learning

• https://www.insidehighered.com/news/2021/02/01/u-illinois-says-goodbye-proctorio

• https://racismandtechnology.center/2021/07/10/call-to-the-university-of-amsterdam-stop-using-racist-proctoring-software/

• https://micky.com.au/proctorio-test-software-fails-to-detect-people-of-color/

• https://www.nytimes.com/2020/09/29/style/testing-schools-proctorio.html

• https://www.pointer.de/studium/aktuelles/17149/pruefungsueberwachung-rassissmus.htm

• https://news.slashdot.org/story/21/04/08/2056257/proctorio-is-using-racist-algorithms-to-detect-faces