Authenticity refers to the quality of something being genuine, original, and what it claims to be. Integrity refers to the wholeness, honesty, and incorruptibility of information, processes, and actors.

AI and automation undermine both in interconnected ways:

• Synthetic & manipulated media. Generative AI enables the creation of convincing fake images, audio, and video that are indistinguishable from genuine content, making it impossible to verify whether what you see or hear is real.

• Identity fraud & impersonation. AI can replicate a person's voice, likeness, or writing style to impersonate them in the form of real-time video calls, automated messages, or fabricated statements.

• Ghostwriting & authorship deception. AI tools allow individuals to pass off machine-generated work as their own, undermining honesty norms in education, publishing, journalism, and professional life.

• Synthetic scientific research. AI has demonstrated the ability to generate high-quality fraudulent scientific papers that are difficult for reviewers and detection tools to identify, raising serious questions about the integrity of published research.

• Provenance uncertainty. As AI-generated content floods the information ecosystem, it becomes increasingly difficult to determine the origin, authorship, or modification history of any given piece of content.

• Undisclosed AI use. Systems, services, or individuals may deploy AI without disclosing it - presenting AI outputs as human-produced work, human interactions as automated ones, or AI-generated news as journalism.

Why it matters

Authenticity and integrity are foundational to how societies function. Trust in information underpins democratic deliberation, journalism, science, law, commerce, and personal relationships. 

When these are corrupted:

• Citizens cannot reliably distinguish real political events from fabricated ones

• Scientific and medical knowledge becomes suspect

• Legal evidence and testimony lose credibility

• Commercial and financial systems become vulnerable to fraud

• Individuals lose the ability to assess the honesty of those they interact with

Content authenticity - the ability to verify the origin, integrity, and history of digital content - is the mechanism by which we confirm that a piece of content is genuine, unaltered, and traceable to a trustworthy source. When AI undermines this, the entire epistemic infrastructure of society is at risk.

Impacts & consequences

Amongst other things, a lack of authenticity and integrity results in: 

• Financial fraud and scams. Advanced social engineering, such as AI voice-cloning used to impersonate family members or corporate executives to authorize fraudulent wire transfers.

• Reputational and psychological damage. The non-consensual creation of explicit deepfakes or defamatory material targeting individuals, particularly women and public figures.

• Academic and scientific degradation. Academic, scientific professional standards can be weakened if AI-generated work is passed off as original human work.

• Political and democratic sabotage. Synthetic leaks, fake speeches, and AI-driven disinformation campaigns that can swing elections or incite violence.

• Erosion of public trust. A pervasive "epistemic nihilism" where citizens stop believing anything they see or hear online.

Causes & contributing factors

The emergence of high volumes of inauthentic content online is due largely to:

• Hyper-realistic generative AI. The democratisation and low cost of powerful open-source models (LLMs, diffusion models, and voice cloners) that require minimal technical skill to operate.

• Engagement-driven algorithms. Social media platforms reward sensationalism and viral engagement, accelerating the spread of unverified or fake AI content.

• Absence of disclosure norms. No universal requirement to label AI-generated content exists.

• Lack of uniform watermarking. A historical and ongoing struggle to enforce universal, tamper-proof digital provenance standards (like C2PA) across all hardware and software.

• Asymmetric detection capabilities. The development of AI generation tools moves at a much faster pace than the development of reliable deepfake/synthetic text detection tools.

• Anonymity and low accountability. Making deception low-risk.