AIAAIC - Report: Denmark AI welfare system "profoundly affects" mental health

Report: Denmark AI welfare system "profoundly affects" mental health

Occurred: November 2024

Report incident 🔥 | Improve page 💁 | Access database 🔢

Denmark's use of artificial intelligence to identify welfare fraud has resulted in an atmosphere of fear, deterioration of mental health and accusations of undue mass surveillance and potentially illegal social scoring.

What happened

Danish welfare authority, Udbetaling Danmark (UDK) extensively employs artificial intelligence (AI) tools to detect potential fraud in a wide range of areas, from pension payments to parental and sick leave, and student grants.

However, analysis of four of the algorithm and reports from individuals subjected to ongoing "mass surveillance" led Amnesty to accuse UDK of "mass surveillance" that infringes on individuals' privacy rights, discriminates against vulnerable groups, and has created a climate of fear and suspicion among those being tracked.

It also argues that the system functions as a social scoring system under the EU's Artificial Intelligence Act and should be banned.

Why it happened

The use of AI in Denmark's welfare system stems from a push for digitisation aimed at identifying fraudulent claims more efficiently and effectively.

The programme has involved the collection of huge amounts of personal data - including residency status, citizenship, family relationships, travel history and health records - and the creation of up to 60 algorithmic models designed to detect social benefits fraud and flag individuals for further investigation.

Amnesty argues that the amount of data collected and processed is "neither necessary nor proportionate". It also says the algorithms used are built on existing societal biases, which exacerbate inequalities rather than alleviate them.

What it means

Amnesty's findings put pressure on Udbetaling Danmark to be consider carefully the workings and impact of its fraud detection systems, and to be more transparent and about how it uses AI.

The report may also lead to discussion in Copenhagen and Brussels about what constitutes a social scoring system.

Artificial intelligence in fraud detection

Artificial intelligence is used by many different businesses and organizations. It is widely used in the financial sector, especially by accounting firms, to help detect fraud.

Source: Wikipedia 🔗

System 🤖

Udbetaling Danmark 🔗

Operator: Arbejdsmarkedets Tillægspension
Developer: Arbejdsmarkedets Tillægspension; NNIT
Country: Denmark
Sector: Education; Govt - welfare
Purpose: Detect fraud
Technology: Classification algorithm; Machine learning
Issue: Accuracy/reliability; Bias/discrimination; Privacy; Surveillance

What happened

Why it happened

What it means

System 🤖

Regulation ⚖️

Research, advocacy 🧮

News, commentary, analysis 🗞️

Related 🌐