Babylon Health Sympton Checker diagnostic chatbot

Report incident 🔥 | Improve page 💁 | Access database 🔢

Incidents and issues 🔥

• First launched in 2016, Babylon's Sympton Checker quickly attracted concerns. A 2017 WIRED assessment of three diagnostic tools rated Babylon's as the least accurate when it came to identifying common illnesses such as asthma and shingles. 

• In June 2018, the company said its chatbot could diagnose common diseases as well as human physicians - a claim that was openly questioned by the Royal College of General Practitioners, the British Medical Association, and the Royal College of Physicians, amongst others.

• A 2018 paper published inThe Lancet concluded that Babylon’s study failed to offer convincing evidence that its symptom checker performs 'better than doctors in any realistic situation, and there is a possibility that it might perform significantly worse.' 

• In February 2020, a BBC Newsnight investigation raised concerns about the bot's safety. And the UK's medical device regulatory body, MHRA, said in a December 2020 letter to NHS consultant Dr David Watkins, a persistent (pseudonymous) critic of the company's governance and products, that his concerns were 'valid' and 'ones that we share'. 

• In September 2019, Dr Watkins accused Babylon's chatbot of possible gender bias in Babylon’s symptom checker - a finding dismissed by Babylon as relevant to the health industry as a whole, but not evident in its diagnostic tool.

Transparency and accountability 🙈

Babylon Health's corporate and product governance were singled out for being inadequate and opaque, and its marketing as misleading. 

It was accused by NHS consultants of rushing to market without adequate proof that its Sympton Checker worked, notably by testing it on real patients, and having failed to conduct any peer-reviewed, randomised control studies before launch. 

Forbes reported in 2018 that 'Interviews with current and former Babylon staff and outside doctors reveal broad concerns that the company has rushed to deploy software that has not been carefully vetted, then exaggerated its effectiveness.'

In February 2020, the company dismissed (pdf) Dr Watkins as a 'troll' and claimed he had 'targeted members of our staff, partners, clients, regulators and journalists and tweeted defamatory content about us.'