What happened

Citing research studies, Motherboard reported that so-called 'automated essay scoring engines' used by 21 US states demonstrated bias against certain demographic groups. 

These included e-rater, an engine developed and run by the nonprofit Educational Testing Service (ETS), which researchers found to have given higher scores to some students, particularly those from mainland China, than did expert human graders. It also tended to underscore African-American students and Arabic, Spanish and Hindi speakers.

Meantime, ACCUPLACER, a machine-scored test owned by the College Board, failed to reliably predict female, Asian, Hispanic, and African-American students’ eventual writing grades.

Furthermore, some systems can be fooled by nonsense essays with sophisticated vocabulary, whilst most stuggle to judge more nuanced aspects of writing, like creativity, experts say.

Why it happened

Automated scoring is proving popular with schools as it is cheaper and faster than human grading.

But concerns about accuracy and fairness persist. These systems use natural language processing (NLP) to predict scores based on patterns from previously graded essays, but they are prone to biases against certain demographic groups.

The algorithms focus on measurable aspects like vocabulary and sentence length, which may disadvantage English language learners and others who write differently.