Study Finds ChatGPT Struggles with Math, Unlikely to Replace Human Accounting Students
Artificial Intelligence language model ChatGPT has been making waves in the education sector due to its ability to complete creative tasks, including storytelling and behavioral learning among others. However, a recent study conducted by Brigham Young University accounting professor, David Wood, has shown that the platform struggles to understand mathematical processes.
The study aimed to test ChatGPT's ability to complete accounting exams compared to actual accounting students. The study administered 25,181 questions on information systems, auditing, financial accounting, managerial accounting, and taxes from 186 educational institutions in 14 countries. The study also received 2,268 textbook test bank questions from undergrad students at BYU. Questions were presented in varying formats including multiple choice, true/false, and written response prompts.
The results of the study showed that the students scored over 30% higher than ChatGPT, with an average score of 76.7% compared to ChatGPT's 47.4%. ChatGPT only outperformed the students on 11.3% of the questions, particularly on those concerning auditing and accounting information systems. The platform was more adept at answering multiple-choice and true/false questions, scoring 59.5% and 68.7% respectively. However, it struggled significantly on short answer questions, only scoring between 28.7% and 39.1%.
While there were initial concerns about ChatGPT being used by students to cheat, David Wood notes that opportunities for cheating have always existed. Instead, he focuses on how the technology can improve the teaching process for faculty and the learning process for students.
In conclusion, while ChatGPT's ability to complete creative tasks is impressive, its struggle with math casts doubt on its potential to replace human accounting students. It is clear that the platform is not perfect and cannot be used for everything. Therefore, students should not rely solely on ChatGPT to learn accounting.
