ChatGPT Beats Doctors at Answering Online Medical Questions, Study Finds
A recent study published in JAMA Internal Medicine suggests that ChatGPT, an AI-powered chatbot, provides better responses to online medical queries than human physicians in terms of quality and empathy. The study evaluated the performance of ChatGPT in answering patient questions from Reddit’s r/AskDocs forum and compared it to that of physicians.
Out of the 195 randomly selected questions involved in the cross-sectional study, chatbot responses were preferred over those of physicians. ChatGPT received significantly higher ratings for both quality and empathy. The study used an older version called GPT 3.5 which suggests that the latest GPT-4 upgrade could provide even better results.
The researchers believe that AI assistants could help draft responses to patient questions, benefiting both clinicians and patients. Chatbots could be used to draft responses for doctors to edit, and randomized trials could assess the potential of AI assistants to improve responses, reduce clinician burnout, and improve patient outcomes.
Prompt and empathetic responses to patient queries could reduce unnecessary clinical visits, free up resources, and promote patient equity. Messaging is also critical for individuals with mobility limitations, irregular work schedules, or fear of medical bills.
However, the study has some limitations. The use of online forum questions may not reflect typical patient-physician interactions. Moreover, the ability of the chatbot to provide personalized details from electronic health records was not assessed, and the accuracy or fabricated information was not evaluated.
Although there are concerns about AI-generated medical advice, the study shows the potential of chatbots in providing healthcare services. Further research is needed to determine the potential impact of AI assistants in clinical settings and address ethical concerns, including the need for human review of AI-generated content for accuracy and potential false or fabricated information.
It’s important to note that ChatGPT is not specifically optimized for medical tasks. Google is developing Med-PaLM 2, a large language model fine-tuned for medical purposes. Google claims it can pass medical exams and plans to start trials with professionals.
