Babylon Health Sympton Checker diagnostic chatbot

Babylon Health's Diagnostic and Triage System (aka 'Sympton Checker') is a chatbot that uses artificial intelligence to ask users questions about their condition in order to identify what is wrong with them and prioritise treatment.

The system has drawn criticism for its perceived poor quality and safety, including alleged errors in diagnosing conditions such as heart attacks, for its opaque governance and misleading marketing claims, and for the manner in which the company responded to the feedback angled at it.

System databank 🔢

Operator: Babylon Health
Developer: Babylon Health

Country: UK; USA; Rwanda

Sector: Health

Purpose: Provide health information

Technology: Chatbot; NLP/text analysis; Deep learning; Neural network; Machine learning
Issue: Accuracy/reliability; Bias/discrimination - gender; Safety

Transparency: Governance; Complaints/appeals; Legal; Marketing

Risks and harms 🛑

Quality, safety

First launched in 2016, Babylon's Sympton Checker quickly attracted concerns. A 2017 WIRED assessment of three diagnostic tools rated Babylon's as the least accurate when it came to identifying common illnesses such as asthma and shingles. 

In June 2018, the company said its chatbot could diagnose common diseases as well as human physicians - a claim that was openly questioned by the Royal College of General Practitioners, the British Medical Association, and the Royal College of Physicians, amongst others.

A 2018 paper published inThe Lancet concluded that Babylon’s study failed to offer convincing evidence that its symptom checker performs 'better than doctors in any realistic situation, and there is a possibility that it might perform significantly worse.' 

In February 2020, a BBC Newsnight investigation raised concerns about the bot's safety. And the UK's medical device regulatory body, MHRA, said in a December 2020 letter to NHS consultant Dr David Watkins, a persistent (pseudonymous) critic of the company's governance and products, that his concerns were 'valid' and 'ones that we share'. 

Bias, discrimination

In September 2019, Dr Watkins accused Babylon's chatbot of possible gender bias in Babylon’s symptom checker - a finding dismissed by Babylon as relevant to the health industry as a whole, but not evident in its diagnostic tool.

Governance, transparency, accountability

Babylon Health's corporate and product governance have been singled out for being inadequate and opaque, and its marketing as misleading. 

It has been accused by NHS consultants of rushing to market without adequate proof that its Sympton Checker worked, notably by testing it on real patients, and having failed to conduct any peer-reviewed, randomised control studies before launch. 

Forbes reported in 2018 that 'Interviews with current and former Babylon staff and outside doctors reveal broad concerns that the company has rushed to deploy software that has not been carefully vetted, then exaggerated its effectiveness.'

In February 2020, the company dismissed (pdf) Dr Watkins as a 'troll' and claimed he had 'targeted members of our staff, partners, clients, regulators and journalists and tweeted defamatory content about us.'

Legal, regulatory

Page info
Type: System
Published: July 2023