Sora - AI video generator

Transparency, accountability 🙈

Sora is seen to suffer from several important transparency and accountability limitations, including:

• Dataset sources. The sources and nature of the training data used to create Sora and Sora 2 have not been disclosed.

• Copyright. Sora 2's launch with an “opt-out default” for copyrighted characters and material means protected content can be generated unless the rights holder actively requests exclusion, leading commentators, lawyers and others to argue it pushes legal responsibility and burdens rather than to OpenAI, therefore minimising its own accountability.

• Technical details. OpenAI has not released comprehensive information about Sora's architecture, training data, or exact capabilities.

• Error rates. Specific data on the frequency and types of errors or artifacts in Sora's output is not readily available.

• Bias mitigation. Information about how potential biases in the training data are addressed is limited.

• Ethical guidelines. Details about the ethical considerations and safeguards implemented in Sora's development and deployment are vague.

• Content moderation. The extent and methods of content moderation for Sora-generated videos are poorly defined.

Risks, harms 🛑

OpenAI's Sora model has been accused of perpetuating a wide variety of risks and harms, despite OpenAI implementing multi-layered safety restrictions around Sora's use, notably to prevent deepfake misuse, harmful content creation and copyright violations. 

These risks and harms include:

• Bias and discrimination. Sora is prone to perpetuating biases present in its training data, including sexist, racist and ableist stereotypes, potentially reinforcing harmful societal prejudices.

• Copyright. OpenAI trained Sora using publicly available videos and copyrighted videos licensed for the purpose, but failed to reveal the number or exact source of the videos, raising questions about abuse of copyright. Sora 2 users have produced mashups and scenes featuring famous characters from Mario to Rick and Morty.

• Privacy and impersonation. Sora relies on huge datasets for training, which may include sensitive or personal information scraped from public profiles or online platforms without explicit consent, raising concerns about data privacy violations and the potential misuse of personal identifiers, such as clips depicting OpenAI CEO Sam Altman shoplifting.

• Physical safety. Sora 2 has been used to stalk and harass a prominent technology journalist.

• Misinformation and disinformation. Sora is regularly misused to create hyperrealistic deepfakes, spread misinformation and produce inappropriate content, raising concerns about its use for harassment, fraud, political and societal manipulation, reputational damage, and other nefarious purposes. Critics also noted that Sora 2’s watermarking and metadata can easily be stripped or cropped, making it hard to detect and counteract harmful deepfakes it has generated.

• Trust in information. The realism of AI-generated videos blurs the line between real and fake content and undermines the credibility of recorded media and public information, thereby eroding societal trust in visual evidence.

• Creativity. Over-reliance on tools like Sora may lead to a homogenised aesthetic or style in creative outputs, reducing diversity in artistic expression.

• Employment. Sora is seen as likely to negatively impact the creative industries, including potentially causing the loss of jobs such as actors, writers and production crews.

• Environment. Given its high energy consumption and associated carbon emissions, Sora and Sora 2 are reckoned likely to cause significant environmental damage.

Legal, regulatory 👩🏼‍⚖️

• Garante (2024). Artificial intelligence: the Italian Data Protection Authority opens an investigation into OpenAI’s ‘Sora’

Research, advocacy 🧮

• Search Google Scholar

News, commentary, analysis 🗞️

• https://www.techradar.com/computing/artificial-intelligence/openai-sora

• https://www.labellerr.com/blog/what-is-openai-sora-how-it-works-everything-need-to-know/