Wisconsin Dropout Early Warning System accused of being 'unreliable'

A system used to predict the likelihood of students graduating at Wisconsin high schools was found to be wrong most of the time, raising questions about the reliability of the system.

Wisconsin's machine learning-based application Dropout Early Warning System (DEWS) was wrong 'almost three-quarters of the time' that it predicted a student would not graduate, and wrong more often about Black and Hispanic students not graduating than about white students, according to a 2023 Markup/ChalkBeat investigation.

The investigation also found that educators had not been properly trained to understand the dropout risk labels and intervene accordingly, and that students had not been informed about the existence of the system.

Meantime, University of California-Berkeley researchers concluded the system failed to achieve its primary goal of improving graduation outcomes for the students it labels as 'high risk'. They recommended Wisconsin scrap its use of individual factors such as a student’s race or even test scores in determining that student’s dropout risk.

Research, advocacy 🧮

• Center for Democracy & Technology (2023). Late Applications. Protecting Students’ Civil Rights in the Digital Age (pdf) 

• Perdomo J.C. et al (2023). Difficult Lessons on Social Prediction from Wisconsin Public Schools

• Christie S. (2019). Machine-Learned School Dropout Early Warning at Scale

• Knowles J. (2015). OF NEEDLES AND HAYSTACKS: BUILDING AN ACCURATE STATEWIDE DROPOUT EARLY WARNING SYSTEM IN WISCONSIN

Investigations, assessments, audits 👁️

• The Markup (2023). Dropout Risk System Under Scrutiny After The Markup Report

• The Markup (2023). Takeaways from Our Investigation into Wisconsin’s Racially Inequitable Dropout Algorithm

• The Markup (2023). False Alarm: How Wisconsin Uses Race and Income to Label Students “High Risk”

News, commentary, analysis 🗞️

• https://pbswisconsin.org/news-item/todd-feathers-on-the-wisconsin-dropout-early-warning-system/

• https://eu.jsonline.com/story/opinion/2023/07/06/rules-to-spur-high-school-graduation-perpetuate-stereotypes/70358424007/

• https://medium.com/@HarlanH/a-critique-of-the-markups-investigation-into-predictive-models-of-student-success-eda01701d0cc

• https://www.caf.com/en/knowledge/views/2021/08/how-does-artificial-intelligence-help-reduce-school-dropouts/