What happened

OpenAI's ChatGPT and text-to-video generator Sora consistently reflect and reinforce caste-based prejudices, according to the MIT Technology Review. A researcher also found caste bias in Sarvam AI, which touts itself as a sovereign AI for India.

The systems were found to generate and perpetuate centuries-old oppressive stereotypes that associate marginalised castes, particularly Dalits, with negative traits such as being impure, uneducated, and limited to menial jobs, while privileging upper castes like Brahmins with positive attributes.

For example, Dhiraj Singha, a sociology researcher from a Dalit background, experienced AI changing his surname to a high-caste name when polishing his academic applications, reaffirming societal biases that define who is deemed “normal” or “fit” for academic success. 

This digital reinforcement of caste-based exclusion is seen to discourage marginalised individuals from pursuing opportunities, and exacerbate economic inequality and social exclusion. 

Why it happened

The root cause lies in AI training on uncurated web-scale data that reflect historical and societal prejudices, including caste-ism, which remain largely unaddressed by developers. 

Despite India being OpenAI's second-largest market, testing for and mitigating caste bias appears not to be a priority, and safety filters have apparently been weakened in the latest GPT-5 model to allow stereotypical outputs rather than refusing to answer problematic queries. 

The AI industry's neglect of non-Western biases like caste - in contrast to more attention on gender and racial bias - is a serious governance and accountability gap.