ChatGPT Lags Behind in Accounting, Discovers Major Study
The latest AI chatbot product, ChatGPT, has failed to meet human standards when it comes to accounting. While it has gained popularity in several other fields, researchers have discovered that AI still has work to do in the accounting realm.
Microsoft-backed OpenAI's newest AI chatbot product, GPT-4, uses machine learning for generating natural language text. It has passed the bar exam with a score in the 90th percentile, 13 of 15 advanced placement (AP) exams, and scored nearly perfectly on the GRE Verbal test.
The researchers, from Brigham Young University (BYU) and 186 other universities, wanted to know how OpenAI's tech would fare on accounting exams. They tested the original version, ChatGPT.
Students scored an average of 76.7%, compared to ChatGPT's average of 47.4%. ChatGPT scored higher than students in 11.3% of questions, performing well in AIS and auditing. However, the AI bot performed poorly on tax, financial, and managerial assessments, possibly because it struggled with the mathematical processes required for these types of questions.
The study, published in Issues in Accounting Education, found that ChatGPT also had difficulty with short-answer questions, and higher-order questions were harder for it to answer. ChatGPT made nonsensical errors in mathematical processes, such as adding two numbers in a subtraction problem or dividing numbers incorrectly.
Furthermore, ChatGPT sometimes produced fabricated references that appeared genuine, including non-existent authors and works.
The authors believe that GPT-4 will improve exponentially. However, the study revealed that chatbots like ChatGPT cannot replace human intellect in the accounting field.
