Epic sepsis prediction model

An algorithm to predict whether or not patients with infections have contracted sepsis has been discovered to have missed about two-thirds of actual cases, rarely found cases medical staff did not notice, and frequently issued false alarms. 

Electronic health record company Epic Systems' Epic Sepsis Model is used by hundreds of hospitals across the US and is marketed as being 76 percent accurate. 

Accuracy, transparency

However, a June 2021 study published in JAMA Internal Medicine by University of Michigan researchers analysing a retrospective sample of over 27,000 adult Michigan Medicine patients concludes the algorithm is only correct 63 percent of the time, and raises many false alarms. 

Part of the problem, Stat News reports, is that the algorithm was trained to flag when doctors would submit bills for sepsis treatment, which doesn’t always line up with patients’ first signs of symptoms. 

In response, Epic pointed to previous research that found the model can accurately predict sepsis, and argued customers have 'complete transparency' into the model. 

In an accompanying editorial, medical researchers argue the findings highlight the need for the external validation of proprietary healthcare prediction models before clinical use.

Data decay

In February 2022, Stat published the findings of a research study conducted with the Massachusetts Institute of Technology that small shifts in data fed into well-known health care algorithms, including the Epic Sepsis Model can cause their accuracy to degrade over time. 

Instead of transforming care, the study finds, the algorithms are unable to keep pace with fast-moving clinical conditions, potentially resulting in mis-diagnoses and raising the prospect AI could do more harm than good.

Epic Systems confirmed in October 2022 that it had overhauled its sepsis prediction model to improve its accuracy and make its alerts more meaningful to clinicians.