ChatGPT shows human-level assessment of brain tumor MRI reports
As artificial intelligence continues to advance, its applications in real-world scenarios are expanding, often matching or even surpassing human capabilities. In the realm of radiology, where accurate diagnoses are paramount for effective patient care, large language models like ChatGPT are proving to enhance accuracy and provide valuable second opinions.
To evaluate its potential, a team led by graduate student Yasuhito Mitsuyama and Associate Professor Daiju Ueda from Osaka Metropolitan University’s Graduate School of Medicine conducted a study comparing the diagnostic performance of ChatGPT based on GPT-4 with that of radiologists. The study analyzed 150 preoperative brain tumor MRI reports, written in Japanese, where ChatGPT, two board-certified neuroradiologists, and three general radiologists were tasked with offering both differential and final diagnoses.
Comparison of Accuracy
The accuracy of each diagnostic entity was then measured against the actual tumor diagnosis post-removal. ChatGPT boasted an accuracy rate of 73%, slightly edging out the 72% average of neuroradiologists and the 68% average of general radiologists. Interestingly, ChatGPT's diagnostic precision varied depending on whether the clinical notes were composed by neuroradiologists or general radiologists. It achieved an 80% accuracy rate with neuroradiologist reports, while its accuracy fell to 60% with general radiologist notes.
Future Implications
Graduate student Mitsuyama remarked, "These findings indicate the potential utility of ChatGPT in the preoperative diagnosis of brain tumors using MRI. We plan to further explore the capabilities of large language models in various diagnostic imaging fields to lighten the load on medical professionals, enhance diagnostic precision, and leverage AI for educational support."
The study detailing these results was published in European Radiology. The research sheds light on the comparative analysis of GPT-4-based ChatGPT's diagnostic efficacy alongside radiologists, utilizing real clinical radiology reports of brain tumors.
For further details, you can access the publication here.
