ChatGPT fails at recommending appropriate cancer treatment

ChatGPT failed to provide an appropriate cancer treatment in approximately one-third of cases, according to researchers. The finding highlighted the need for increased awareness of the system's limitations for medical use.

Focusing on the three most common cancers (breast, prostate and lung cancer), researchers from Brigham and Women's Hospital used 104 prompts to get ChatGPT to provide a treatment approach for each cancer based on the severity of the disease, with the aim of evualting how consistently ChatGPT provided recommendations for cancer treatment that aligned with US National Comprehensive Cancer Network (NCCN) guidelines. 

They found that nearly all responses (98 percent) included at least one treatment approach that agreed with NCCN guidelines. However, the researchers found that 34 percent of these responses also included one or more inappropriate ('non-concordant') recommendations, which were sometimes difficult to detect amidst otherwise sound guidance. 

They also discovered that ChatGPT produced 'hallucinations,' or a treatment recommendation entirely absent from NCCN guidelines, in 12.5 percent of cases. These included recommendations of novel therapies, or curative therapies for non-curative cancers.