ChatGPT Answers Patients' Questions Better Than Doctors: Study
ChatGPT, an AI language model developed by OpenAI, provided better answers to patient queries than human doctors in four out of five instances, according to a recent study published in JAMA Internal Medicine. The study's lead author, John W. Ayers, said that given the paucity of medical professionals, the study's results have major implications for the future of healthcare. "Doctor inboxes are filled to the brim after this transition to virtual care because of COVID-19. Patient emails go unanswered or get poor responses, and providers get burnout and leave their jobs. With that in mind, I thought ‘How can I help in this scenario?’" said Ayers.
A panel of licensed healthcare professionals with backgrounds in internal medicine evaluated the exchanges. The panel first chose which response was they thought was better, and then evaluated both the quality of the answers and the empathy or bedside manner provided. ChatGPT’s answers were rated “good” or “very good” more than three times more often than doctors’ responses. The AI was rated “empathetic” or “very empathetic” almost 10 times more often.
The study pulled from Reddit's r/AskDocs, where doctors with verified credentials answer users' medical questions. The study randomly collected 195 questions and answers, and then had ChatGPT answer the same questions. The results showed that the AI provided high-quality responses and was more empathetic than human doctors.
While the results seem promising, the medical community needs to move with caution. AI is advancing quickly, and as the technology advances, so do the potential harms. The study's lead author said, "We're not saying we should just flip the switch and implement this. We need to pause and do the phase one studies, to evaluate the benefits and discover and mitigate the potential harms."
Applied properly, a system that uses AI chatbots in medicine can even be a tool to combat the epidemic of misinformation and disinformation that is probably the single biggest threat to public health today. However, if not designed properly, it may make misinformation and disinformation even more rampant.
