Occurred: November 2023
Cruise knew that its self-driving cars had been struggling to recognise children, but kept its vehicles on the road anyway.
According to internal safety assessment documents obtained by The Intercept, Cruise was concerned that its vehicles might drive too fast at crosswalks or near a child moving abruptly into the street. The materials also indicate Cruise lacks data around scenarios such as kids suddenly separating from their accompanying adult, falling down, riding bicycles, or wearing costumes.
The materials also revealed that Cruise lacked high-precision machine learning software that would automatically detect child-shaped objects around the car and manoeuvre accordingly, and was relying on human beings to manually identify children encountered by AVs where its software could not do so automatically.
Cruise responded that its vehicles have had 'no on-road collisions with children'.
Operator: General Motors/Cruise LLC
Developer: General Motors/Cruise LLC
Purpose: Automate steering, acceleration, braking
Technology: Self-driving system; Computer vision; Machine learning
Transparency: Governance; Safety
Investigations, assessments, audits
News, commentary, analysis
Published: November 2023