AI, algorithmic and automation harms taxonomy

Artificial intelligence, algorithmic, and automation systems are increasingly central to the everyday operation of government, business and society. However, despite a surge of incidents and controversies, heightened public awareness and interest, and the prospect of dedicated regulation, many of these systems remain opaque, and their impacts difficult to understand.

AIAAIC has developed a general purpose taxonomy of harms which will be refined over the coming months in a structured and transparent manner involving a large, diverse group of individuals representing a broad range of interests and expertise, countries and cultures, ages and genders.

Objectives

AIAAIC's harms taxonomy project aims to: 

• Develop a general purpose, open, human-centered taxonomy of real-world harms of AI, algorithms and automation

• Equip researchers, civil society organisations and the general public to better understand and take action on AI and related technology harms and violations.

The taxonomy is intended to be:

• Clear. Simple categories and definitions that are understandable to multiple audiences

• Comprehensive. Reflective of a wide range of harms - though not necessarily exhaustive

• Flexible. Adaptable to new harms as they arise

• Interoperable. Can connect and communicate in real-time with other systems and organisations

Audiences

The harms taxonomy is intended to be relevant to the following audiences: 

• Researchers and academics

• NGOs, journalists, teachers, and other civil society entities

• Customers, consumers, and the general public

• Policymakers and regulators

• Businesses.

Approach

AIAAIC and its partners will use the following principles to inform the development of phase 2 of the harms taxonomy project:

1. Transparent. About our objectives, processes, outcomes, etc

2. Collaborative. Meaningfully involving participants in taxonomy development and decision-making

3. Inclusive. Of a broad range of expertise, genders, ages, nationalities, races and ethnicities

4. Rigorous. Taking a structured, evidence-based approach to all aspects of the project, including annotation testing.

Methodology

Phase 2 of the harms taxonomy project is envisaged to consist of: 

• Expert outreach & review. Identify and involve experts with deep knowledge of key harm categories to review and refine the taxonomy on an ad hoc, as needed basis.

• Student network outreach & review. Reach out to students and others interested in AI and related technologies but who are not considered experts, via established communities and networks to review and refine the taxonomy.

• Annotation testing & review. Select and annotate 200-300 entries to the AIAAIC Repository representative of a broad range of scenarios and annotate using a panel of testers via the annotation testing tool. Each wave of testing will be reviewed and the taxonomy updated by the project Steering Committee.

• End user/subject review. The taxonomy will be reviewed for clarity and usability by a diverse selection of system end users/subjects.

Outputs

The outputs from the project are envisaged to be:

• Taxonomy and definitions

• Machine-readable version

• Harms dataset

• Research paper

• White paper.