IBM accused of secretly using NYPD images to train object recognition tool

Occurred: September 2018

IBM developed a surveillance tool for the New York Police Department (NYPD) that could identify individuals based on physical characteristics, including skin colour, raising serious privacy and racial profiling concerns.

What happened

IBM secretly used video footage collected by the NYPD to develop software capable of identifying individuals based on features such as hair colour, clothing colour, facial features and skin tone. 

The system could recognise more than 16,000 data points, making it highly accurate for recognising faces. 

While the NYPD claimed the skin tone search feature was only used for evaluation purposes, the technology's development and potential use raised significant ethical concerns about privacy and racial profiling.

Why it happened

The NYPD began collecting images of New Yorkers following the 9/11 attacks to monitor potential terrorist activity and other suspicious behaviour. 

IBM partnered with the NYPD to develop object identification technology, gaining access to the department's camera footage to create new search features for law enforcement use. 

What it means

IBM's activities represent a significant breach of privacy, as thousands of unknowing New Yorkers had their images used to develop surveillance technology without their knowledge or consent. 

Furthermore, the potential for misuse and racial profiling is substantial, given the system's ability to search based on skin tone and ethnicity.

For society, this case highlights the urgent need for regulation and transparency in the development and use of facial recognition and surveillance technologies. It raises critical questions about the balance between public safety and individual privacy rights, as well as the potential for these technologies to exacerbate existing racial biases in law enforcement.

The incident underscores the importance of public oversight and ethical considerations in the deployment of AI-powered surveillance tools in urban environments.

System 🤖

Operator: 
Developer: IBM
Country: USA
Sector: Govt - police
Purpose: Identify facial features, skin tone, age, etc
Technology: Object identification
Issue: Privacy/surveillance; Transparency