SQA algorithm downgrades 124,000 predicted exam results

Occurred: August 2020-

Report incident 🔥 | Improve page 💁 | Access database 🔢

An algorithm used by the Scottish Qualifications Authority (SQA) faced significant criticism due to its handling of exam predictions.

Developed by the SQA, the algorithm was introduced due to restrictions imposed by the Scottish government during the COVD-19 pandemic which meant that in-person exams had to be scrapped.

The new system was widely criticised by students, parents, teachers, education experts and politicians after:

The SQA rejected nearly 124,000 recommendations from teachers for exam results, downgrading nearly a quarter of student submissions. Approximately 40 percent of students received scores lower than their teachers’ predictions, impacting their university prospects. Only 7 percent of entries were revised upwards.

Students from working-class and disadvantaged backgrounds were found to have been disproportionately affected by severe downgrades compared to their privileged peers, thereby exacerbating socio-economic disparities. The SQA cut Higher passes for pupils from the most deprived areas by over 15 points, from the 85.1 percent pass rate recommended by teachers down to 69.9 percent. By comparison, grades were cut in only 9.8 percent of cases for the best off pupils.
5 percent of schools did not receive student results.

The incident was seen to have led to a loss of confidence and trust in Scotland's education system and government. SQA was also taken to task for a perceived lack of transparency, including its refusal to publish details of the algorithm. Several SQA management figures including its Chief Executive, Ron Tuck, resigned or lost their jobs as a result of the crisis.

System 🤖

Operator: Education Scotland
Developer: Scottish Qualifications Authority
Country: UK - Scotland
Sector: Education
Purpose: Predict exam grades
Technology: Prediction algorithm
Issue: Accuracy/reliability; Bias/discrimination - economic
Transparency: Governance; Complaints/appeals; Legal

Research, advocacy 🧮

Kippin S., Cairney P. The COVID-19 exams fiasco across the UK: four nations and two windows of opportunity

Investigations, assessments, audits 🧐

Office for Statistics Regulation (2021). Ensuring statistical models command public confidence (pdf)
Scottish Government (2020). National Qualifications experience 2020: rapid review

System 🤖

Research, advocacy 🧮

Investigations, assessments, audits 🧐

News, commentary, analysis 🗞️