Niulanshan First Secondary School Classroom Care System opacity

The movements and behaviour of students at Niulanshan First Secondary School in Beijing are being constantly monitored and scored by a system about which they have been no given information nor to which they have provided their consent. 

Launched in 2017, Hanwang Education's Class Care System (CCS) uses facial recognition, emotion recognition and deep learning algorithms to identify students' faces and analyse and classify their behaviour into one of five categories: listening, answering questions, writing, interacting with other students, or sleeping.

Each student is automatically scored weekly and their information shared via a mobile app with their teachers and parents. 

Hanwang argues the scores are important for identifying which students require support. But critics point out that the system is not only highly intrusive, it can also be inaccurate and unfair. 

As Rest of World describes, in one example 'a student who had answered just a single question in his English class was called out for low participation — despite the app recording him as 'focused' 94% of the time.'

Research studies regularly conclude that emotion recognition technologies are based on the fundamentally flawed premise that algorithms can analyse a person’s facial expressions and accurately infer their inner state or mood.

System

• Hanwang Technology website

Research, advocacy

• 5 Rights Foundation/Digital Futures Commission (2022). Automated empathy in education: benefits, harms, debates

• Article 19 (2021). Emotional Entanglement: China’s emotion recognition market and its implications for human rights (pdf)

• Durán J. I., Reisenzein R., Fernández-Dols J-M. (2017). Coherence Between Emotions and Facial Expressions: A Research Synthesis

News, commentary, analysis

• https://thedisconnect.co/three/camera-above-the-classroom/

• http://www.sixthtone.com/news/1003759/camera-above-the-classroom

• https://www.caixinglobal.com/2019-03-30/pilot-programs-are-spying-on-kids-in-the-classroom-101398890.html

• https://www.caixinglobal.com/2019-04-06/pilot-programs-are-spying-on-kids-in-the-classroom-part-two-101401105.html

• https://restofworld.org/2021/chinas-emotion-recognition-tech/