France welfare fraud detection algorithm accused of exacerbating inequality
Occurred: December 2023
Report incident 🔥 | Improve page 💁 | Access database 🔢
A welfare fraud detection algorithm used by France's social security agency, CNAF, was accused of exacerbating inequality and targeting the most vulnerable populations.
Introduced in the late 2000s to detect fraud, the algorithm was found to disproportionately targets vulnerable groups, such as single parents, low-income individuals, and people with disabilities, who have no apparent connection to fraud, according to a dual investigation by Lighthouse Reports and Le Monde. It was used to score almost half of France's population.
According to the investigation, the algorithm's criteria were poorly constructed, leading to arbitrary and potentially unfair flagging of individuals based on minor changes in behaviour. People flagged by the algorithm were subjected to invasive investigations, including searches of their homes and scrutiny of personal records, an experience described as distressing and stigmatising.
Despite its public domain use, the algorithm remained unknown to French citizens and others, and its criteria for scoring remained largely undisclosed. Efforts by digital rights groups to obtain the source code of the algorithm faced significant resistance, with crucial variables redacted, limiting the ability to fully understand how the algorithm operates.
➕ October 2024. La Quadrature du Net and fourteen coalition partners submitted a complaint (pdf) to France's Council of State demanding CNAF's risk-scoring algorithmic system is stopped.
System 🤖
Operator: Caisse Nationale des Allocations Familiales (CNAF)
Developer: Caisse Nationale des Allocations Familiales (CNAF)
Country: France
Sector: Govt - welfare
Purpose: Detect fraud
Technology: Risk assessment algorithm; Machine learning
Issue: Accountability; Accuracy/reliability; Bias/discrimination - disability; Transparency
Legal, regulatory 👩🏼⚖️
Le Quadrature du Net. Legal complaint (pdf)
Research, advocacy 🧮
Amnesty. France: Discriminatory algorithm used by the social security agency must be stopped
Pulitzer Center. How We Did It: Unlocking Europe's Welfare Fraud Algorithms
Le Quadrature du Net. SCORING OF WELFARE BENEFICIARIES: THE INDECENCY OF CAF’S ALGORITHM NOW UNDENIABLE
Le Quadrature du Net. NOTATION DES ALLOCATAIRES : L’INDÉCENCE DES PRATIQUES DE LA CAF DÉSORMAIS INDÉNIABLE
Investigations, assessments, audits 🧐
Lighthouse Reports. France's digital inquisition
Lighthouse Reports. How We Investigated France’s Mass Profiling Machine
News, commentary, analysis 🗞️
Page info
Type: Incident
Published: July 2024
Last updated: October 2024