What happened

A 2017 BBC investigation discovered hundreds of videos of well-known children's cartoon characters on YouTube and YouTube Kids, including Peppa Pig, PAW Patrol, Doc McStuffins, and Thomas the Tank Engine. 

These videos incorporated creepy and disturbing content that passed for real cartoons when viewed by kids, including animated violence and graphic toilet humour, and had not been detected by YouTube's screening software partly as their creators had used animation and keywords targeting children to circumvent it. 

Young children were exposed to graphic violence, simulated abuse, medical torture, and fear-inducing imagery that was developmentally inappropriate. Reported effects included nightmares, anxiety, distress, and behavioural changes, with particular risk to very young viewers who lack the cognitive ability to distinguish parody or fiction from reality. 

YouTube responded by advising parents to use its YouTube Kids app and turn on 'restricted mode'. It also removed some of the videos flagged by the BBC. Months later, the company introduced a policy that age restricted content deemed an inappropriate use of family cartoon characters on the YouTube main app when flagged. 

What it happened

The incident constituted a massive failure in automated content moderation and a lack of corporate transparency regarding how children's content was vetted.

• Algorithmic blindness: YouTube relied almost exclusively on AI classifiers to sort content. These algorithms could recognise the "pixels" of Peppa Pig but could not understand the "context" or "narrative" of the violence taking place.

• Financial incentives: YouTube’s recommendation engine prioritised engagement and watch time. Because these videos were weird and high-energy, they kept children staring at screens, which the algorithm interpreted as "successful" content, subsequently rewarding creators with significant ad revenue.

• Lack of accountability: Critics argued that YouTube treated its platform as a "neutral" host rather than a publisher with a duty of care. Until the public outcry, there was little proactive human review of the "YouTube Kids" ecosystem, which many parents assumed was a curated, "walled garden."