IBM accused of secretly using NYPD images to train object recognition tool
IBM accused of secretly using NYPD images to train object recognition tool
Occurred: September 2018
Report incident 🔥 | Improve page 💁 | Access database 🔢
IBM developed a surveillance tool for the New York Police Department (NYPD) that could identify individuals based on physical characteristics, including skin colour, raising serious privacy and racial profiling concerns.
IBM secretly used video footage collected by the NYPD to develop software capable of identifying individuals based on features such as hair colour, clothing colour, facial features and skin tone.
The system could recognise more than 16,000 data points, making it highly accurate for recognising faces.
While the NYPD claimed the skin tone search feature was only used for evaluation purposes, the technology's development and potential use raised significant ethical concerns about privacy and racial profiling.
The NYPD began collecting images of New Yorkers following the 9/11 attacks to monitor potential terrorist activity and other suspicious behaviour.
IBM partnered with the NYPD to develop object identification technology, gaining access to the department's camera footage to create new search features for law enforcement use.
IBM's activities represent a significant breach of privacy, as thousands of unknowing New Yorkers had their images used to develop surveillance technology without their knowledge or consent.
Furthermore, the potential for misuse and racial profiling is substantial, given the system's ability to search based on skin tone and ethnicity.
For society, this case highlights the urgent need for regulation and transparency in the development and use of facial recognition and surveillance technologies. It raises critical questions about the balance between public safety and individual privacy rights, as well as the potential for these technologies to exacerbate existing racial biases in law enforcement.
The incident underscores the importance of public oversight and ethical considerations in the deployment of AI-powered surveillance tools in urban environments.
Unknown
Operator:
Developer: IBM
Country: USA
Sector: Govt - police
Purpose: Identify facial features, skin tone, age, etc
Technology: Object identification
Issue: Privacy/surveillance; Transparency
Page info
Type: Issue
Published: February 2025