ChatGPT Outshines Physicians in Quality and Empathy | OBN
A recent study published in JAMA Internal Medicine showcased the effectiveness ChatGPT, an AI-based chatbot assistant, in answering patient queries on a publicly available social media forum. The study compared the quality and empathy of ChatGPT versus that of physicians in responding to patient questions.
In the study, a team of researchers randomly selected 195 exchanges from a publicly accessible social media forum, r/AskDocs on Reddit, in October 2022. The team used the original full text of a patient's question to generate a new chatbot session and compared the responses of the physician and the chatbot. The responses were evaluated by licensed healthcare professionals and were anonymized to protect patient identities and comply with the Health Insurance Portability and Accountability Act (HIPAA).
The results showed that evaluators preferred ChatGPT responses over physician responses, with 78.5% of ChatGPT responses rated 'good' or 'very good' for quality compared to 22.1% for physicians. Moreover, ChatGPT responses were rated significantly higher in terms of empathy than even the longest responses authored by physicians. Specifically, 45.1% of ChatGPT responses were rated 'empathetic' or 'very empathetic' compared to 4.6% for physicians.
Utilizing AI assistants like ChatGPT to provide medical advice to patients on social media forums could potentially save clinical staff time, provide more consistent responses, and allow staff to focus on more complex tasks. Furthermore, patients who receive quick responses to their inquiries may be less likely to schedule unnecessary clinic visits and could potentially improve health behaviors, such as adhering to a strict diet and medication regimen.
However, randomized clinical trials are necessary to evaluate the efficacy of AI-based technologies before implementing them in real-world clinical settings. Furthermore, such trials should examine the impact on clinical staff and physician burnout in greater depth. Overall, this study highlights the potential benefits of AI assistants for improving both clinician and patient outcomes.
