Clinicians Agree With Some ChatGPT Responses to Patient Inquiries About Colon Cancer
Artificial intelligence (AI) systems like ChatGPT are proving to be valuable resources in providing accurate and relevant responses to inquiries about colon cancer. A recent study presented at the ASCO Gastrointestinal Cancers Symposium 2025 highlighted the potential of AI chatbots in enhancing patient education in oncology.
Assessing ChatGPT Responses
Dr. Sujeta Ojha of the Dell Medical School, The University of Texas at Austin, led a research team to evaluate the accuracy of ChatGPT responses to patient inquiries regarding colon cancer. The team compared responses from ChatGPT with assessments provided by expert clinical oncologists.
Research Methodology
The research team curated a set of 10 comprehensive questions based on commonly asked queries from reputable sources like cancer institutes and societies. These questions covered general oncology characteristics, including symptoms, screening, prevention, as well as diagnosis and treatment aspects.
The questions were input into the ChatGPT AI model, and the responses were evaluated by oncology experts for accuracy and relevance. Experts rated the responses on a 5-point Likert scale, with 1 indicating “strongly disagree” and 5 indicating “strongly agree.”
Study Findings
The overall mean score for the ChatGPT responses was 4.72. While the analysis did not reveal any significant variance in mean scores among raters, experts tended to agree less with responses related to “diagnosis and treatment” (mean score of 4.52) compared to those on “general oncology characteristics” (mean score of 4.92).
The study suggests that ChatGPT can be a valuable tool in providing patients with informative content on colon cancer, potentially complementing traditional medical education efforts. However, experts showed a significant difference in agreement with ChatGPT's responses on diagnosis and treatment, possibly due to the complexity of clinical decision-making and evolving treatment protocols in oncology.
Future Directions
The researchers noted that future iterations of the project could involve input from experts in related fields such as Gastroenterology and Colorectal Surgery to provide a more comprehensive assessment of ChatGPT’s responses. This would enhance the applicability of AI in various facets of colon cancer care.
