ChatGPT's Limitations: Experts Warn It May Outdo...
ChatGPT, the AI-based language processing model, has become renowned for achieving human-like results on language tasks. However, a recent study published in the American Accounting Association journal reveals that ChatGPT has not yet mastered accounting subjects as human learners have.
The study, conducted by 327 authors from 186 institutions across 14 countries, tested ChatGPT's accounting proficiency with 25,181 classroom accounting exam questions and 2,268 textbook test bank questions, revealing that it performed worse than undergraduate students. The results showed students scored an average of 76.7%, whereas ChatGPT scored only 47.7%.
Despite providing authoritative written answers to some questions that turned out to be incorrect, ChatGPT outperformed students in specific accounting subjects such as information systems and auditing. But it struggled with mathematical aspects involved in taxes, financial, and managerial assessment. Furthermore, it performed better on true or false and multiple-choice questions, but struggled with short-answer questions and higher-order questions.
David Wood, the lead author of the study and a professor of accounting at Brigham Young University, explained the focus of the study was not mainly on stopping students from cheating. Instead, it sought to establish ChatGPT's usefulness in enhancing the teaching process for faculty and students. The results, while not perfect, demonstrated the potential to improve ChatGPT's accounting proficiency in the future.
Experts warn that AI technology has limitations, and ChatGPT's failure to outdo human learners in accounting exams is proof of these limitations. However, the study authors are optimistic that GPT-4 could solve the issues that ChatGPT struggled with, performing much better on accounting exams.
