AI Chatbot ChatGPT Fails in Accounting: Study
Researchers at various universities, including Brigham Young University, put AI chatbot ChatGPT to the test in accounting exams. ChatGPT is an AI chatbot product launched by OpenAI, which uses machine learning to generate natural language text. The researchers found that while the AI chatbot performed impressively on some questions, it did worse than human students on others.
ChatGPT vs Human Students
According to the study, human students scored an overall average of 76.7%, while ChatGPT scored 47.4%. The AI chatbot performed well on Accounting Information System (AIS) and auditing questions, but struggled with tax, financial, and managerial assessments. The researchers found that ChatGPT performed better on true/false questions and multiple-choice questions, but struggled with short-answer questions and higher-order questions. ChatGPT's lack of understanding of mathematical processes required for some questions may be the reason for its poor performance, the study suggested.
ChatGPT Makes Nonsensical Errors
The study found that ChatGPT often provided explanations for its answers, even if they were incorrect. Additionally, ChatGPT sometimes made up facts, which were not even remotely accurate. The authors, however, expect GPT-4 to improve significantly on accounting questions posed in their study.
Overall, while the AI chatbot ChatGPT has shown promise in various domains, it still has work to do in the realm of accounting.
