What happened

Since at least 2013, Sweden’s Social Insurance Agency (Försäkringskassan) used a machine learning algorithm to assign risk scores to welfare applicants, automatically flagging those deemed “high risk” for fraud investigations.

However, investigations by Lighthouse Reports and Svenska Dagbladet, supported by Amnesty International, revealed that the system unjustly targets marginalised groups, and especially women and people with foreign backgrounds, at significantly higher rates than others, even when there is no evidence of wrongdoing.

Those flagged by the algorithm often face invasive investigations, delays, or suspension of benefits, sometimes leaving them unable to cover basic needs and causing psychological and financial harm and eroding trust in Sweden’s welfare system.

The process is opaque: applicants are not informed of their risk scores or the reasons for being flagged, and the agency has refused to fully disclose how the algorithm works. 

Why it happened

The bias stems from the algorithm’s design and the data used to train it. The model incorporates demographic and behavioural data, but analysis shows it systematically overrepresents women, migrants, low-income earners, and those without higher education among those flagged for fraud. 

Critics argue that these disparities are the result of embedded bias in the algorithm’s construction and the agency’s flawed methodology for estimating fraud risk. 

Despite warnings from Sweden’s Inspectorate for Social Security in 2018 and ongoing criticism, the agency has denied wrongdoing and has not implemented effective safeguards against discrimination.