Students Score Higher than ChatGPT on Accounting Exams, Say Researchers
According to recent research published in the journal Issues in Accounting Education, students scored higher on accounting exams than OpenAI's ChatGPT, which is a chatbot product that uses machine learning to generate natural language text. Despite this, the researchers from Brigham Young University (BYU), as well as 186 other universities, said that ChatGPT's performance was "impressive" and that it had the potential to change the way we teach and learn in a positive way.
The students scored an average of 76.7 per cent in the accounting exam, while ChatGPT scored an average of 47.4 per cent. ChatGPT performed better on accounting information systems (AIS) and auditing, but struggled with tax, financial, and managerial assessments, possibly due to the mathematical processes required for the latter. The AI bot also performed better on true/false questions (68.7 per cent correct) and multiple-choice questions (59.5 per cent) but fared poorly with short-answer questions (between 28.7 and 39.1 per cent).
The researchers found that higher-order questions were more challenging for ChatGPT to answer. Sometimes, the AI bot would provide authoritative descriptions for incorrect answers and answer the same question differently. ChatGPT often gave explanations for its answers, even if they were incorrect. In some instances, ChatGPT made up facts and generated fake references. It also made nonsensical mathematical errors such as adding two numbers in a subtraction problem or dividing numbers incorrectly.
The lead author of the study, David Wood, recruited 327 co-authors from 186 educational institutions in 14 countries, who contributed 25,181 classroom accounting exam questions for the research. Additionally, undergraduate BYU students provided another 2,268 textbook test bank questions for ChatGPT to answer, covering AIS, auditing, financial accounting, managerial accounting, and tax questions of varying difficulty and type (true/false, multiple choice, and short answer).
The researchers argued that although ChatGPT's performance was not perfect, the AI bot had the potential to enhance education by providing faster feedback, freeing up teachers' time, and helping students learn at their own pace.
Follow us on HT Tech for the latest tech news and reviews. Stay up to date with us on Twitter, Facebook, Google News, and Instagram. Subscribe to our YouTube channel for the latest videos.
