Representation problems begin with data: if training data under-represents certain genders, ethnicities, ages, accents, disabilities, or regions, the system may perform worse for them or treat them as exceptions. 

It also includes design choices, such as whose needs are prioritised, which languages are supported, and whether people affected by the system had any input into how it was built.

Representation also affects outputs. An AI system can mislabel people, present stereotyped images or text, or treat one group as the default “normal” user while others are handled poorly or invisibly. 

In automation, the same issue appears when workflows, forms, or decision rules assume a narrow range of bodies, identities, work patterns, or life circumstances.

Why it matters

Representation matters because AI and automation increasingly shape access to jobs, healthcare, education, media, finance, and public services. Missing or misrepresented groups can receive worse outcomes, less accurate decisions, or fewer opportunities.

It also matters socially because repeated distortions can normalise stereotypes and weaken trust in AI systems and institutions. In the long run, poor representation can make automation less reliable, less inclusive, and less legitimate.

Impacts & consequences

The real-world consequences can be severe and widespread when representations norms break down. 

• Stereotyping. Stereotypes are amplified and normalised at scale across millions of interactions and outputs. A 2023 analysis of more than 5,000 images created with AI image generator Stable Diffusion found that it simultaneously amplifies both gender and racial stereotypes.

• Economic exclusion. Underrepresented groups are filtered out of employment, credit, and opportunity by biased algorithmic systems.

• Reputational damage. Misrepresentation - particularly the sexualisation or caricature of women and people of colour - can cause direct psychological and reputational harm.

• Compounding injustice. AI systems trained on historical data that reflects existing bias can systematically flag members of one group as higher risk, reproducing and entrenching systemic inequality.

• Distortion of the historical and cultural record. Generative AI tools can produce anachronistic or inaccurate depictions of history, distorting public understanding of the past.

• Feedback loops. Biased outputs become new training data, deepening existing imbalances over time.

Causes & contributing factors

The root causes of representation issues in AI are rarely malicious intent; rather, they stem from systemic oversights.

• Homogeneous training data. Drawn from sources (internet text, historical hiring records, legal databases) that overrepresent majority groups and reflect systemic, historical biases and unequal power structures.

• Homogeneous tech teams. A lack of diversity among AI developers, data scientists, and executives leads to blind spots regarding how technology affects different communities.

• Inadequate auditing and testing. Across demographic groups before deployment of an AI system. 

• Profit-driven shortcuts. That prioritise speed and scale over inclusive data collection.