ChatGPT Outperforms Doctors In Answering Patient Messages
The healthcare sector has been notoriously slow to adopt new technologies in the past, but it certainly hasn’t been ignoring ChatGPT, an AI-powered model that has caught the attention of healthcare leaders. ChatGPT has potential use cases, such as assisting with clinical note-taking and generating hypothetical patient questions to which medical students can respond. Recently, healthcare software giant Epic announced that it will integrate GPT-4, the latest version of the ChatGPT model, into its electronic health record.
Meanwhile, researchers are hard at work measuring ChatGPT’s ability to alleviate healthcare’s workforce and burnout crises. A new study published in JAMA Internal Medicine found that ChatGPT might actually be quite successful in providing high-quality, empathetic answers to patient questions during an era in which doctors and nurses are too busy to do so.
The study compared two sets of written responses to real-world patient questions. One set was written by physicians, the other by ChatGPT. The patient questions were sourced from Reddit’s AskDocs, a subreddit with about 452,000 members who ask medical questions and receive answers from licensed medical professionals. Both sets of answers were evaluated by a panel of licensed healthcare professionals.
The results were unexpected as the panel preferred ChatGPT’s responses 79% of the time. More than a quarter of physicians’ responses were deemed as less than acceptable in quality, while this was the case for only 3% of ChatGPT responses. The panel also rated ChatGPT’s responses as more empathetic. Nearly half of the AI model’s responses were categorized as empathetic, while just 5% of physicians’ responses were.
It might be surprising that responses from an AI model ended up being deemed more empathetic than those given by humans, but the study found that ChatGPT's ability to express empathy is due to physicians being too busy or burnt out to provide empathetic responses. ChatGPT does not have a crowded schedule and is not suffering from burnout, which enables it to express empathy in its responses.
The study demonstrated that ChatGPT has significant potential to alleviate the massive burden physicians face in their inboxes. AI assistants help manage workflows and communication with patients, improving patient outcomes. The model can provide faster, more detailed responses to patient questions, so it can boost patients’ health by helping them better manage their conditions at home. However, more clinical studies are required to prove ChatGPT's ability to improve patient outcomes.
