ChatGPT incorrectly diagnoses most pediatric cases

ChatGPT incorrectly diagnosed over 8 in 10 pediatric case studies, according to a new research study. The findings raised questions about the chatbot's suitability as a diagnostic tool for complex conditions. 

Researchers at Cohen Children's Medical Center, USA, pasted the text of 100 pediatric case challenges published in JAMA Pediatrics and the New England Journal of Medicine between 2013 and 2023 into ChatGPT, with two qualified physician-researchers scoring the AI-generated answers as correct, incorrect, or 'did not fully capture the diagnosis.' 

ChatGPT got the right answer in 17 of the 100 cases, was wrong in 72 cases, and failed to fully capture the diagnosis of the remaining 11 cases. Among the 83 wrong diagnoses, 47 (57 percent) were in the same organ system. 

The findings suggest ChatGPT should not currently be used to assess complex pediatric cases, and that 'more selective training' is required to make it more accurate and reliable. The authors also suggest that chatbots could improve with more real-time access to medical data.