ChatGPT falls short in accounting exams. Study reveals students...
A recent study conducted by researchers at Brigham Young University and 186 other universities revealed that OpenAI's ChatGPT chatbot product falls short in accounting exams when compared to students. However, the study did find that ChatGPT's performance was impressive and had the potential to revolutionize the way people teach and learn.
In the study, accounting students scored an average of 76.7% on the exam, while ChatGPT scored only 47.4%. While ChatGPT outperformed the student average in 11.3% of the questions, it struggled with certain areas such as tax, financial, and managerial assessments, possibly due to its difficulty with mathematical calculations.
ChatGPT utilizes machine learning to produce natural language responses and performed better on true/false questions (68.7% correct) and multiple-choice questions (59.5%) but struggled with short-answer questions (with correct answers ranging from 28.7% to 39.1%). The chatbot also had difficulty with higher-level questions, sometimes providing detailed responses for incorrect answers or responding inconsistently to the same question.
Despite these shortcomings, the study's lead author, David Wood, enlisted numerous professors to compare ChatGPT's performance with that of university accounting students. Students from various universities provided ChatGPT with 2,268 questions from various accounting domains of varying complexity and types, including AIS, auditing, financial accounting, managerial accounting, and tax.
In conclusion, the study showed that while ChatGPT falls short in accounting exams when compared to accounting students, its performance was still impressive and had the potential to revolutionize the way people teach and learn.
