Testing the capacity of Bard and ChatGPT for writing essays on ...
Recent advancements in artificial intelligence (AI) have raised concerns about its potential misuse, particularly in the field of education. Large language models (LLMs) such as ChatGPT and Bard have the ability to generate human-like text, leading researchers and educators to question the necessity of assigning students with writing academic essays.
To investigate this further, we conducted a study to explore whether ChatGPT and Bard could produce unstructured essays on medical students' personal experiences of challenges and ethical dilemmas that closely resembled those written by humans. We collected 47 original student-written essays and extracted keywords to create prompts for the LLMs. Using these prompts, we generated an equivalent number of essays with ChatGPT and Bard.
Analysis Process
We analyzed the essays using the Language Inquiry and Word Count (LIWC) 22 software, focusing on extracting main LIWC summary measures and variables related to social and psychological processes. Additionally, we performed sub-analyses on sixteen student essays that were suspected to be partially or fully written by AI, as detected by two AI detectors.
Key Findings
Our study revealed that AI-written essays exhibited a higher usage of language related to affect, authenticity, and analytical thinking compared to student-written essays, once AI-co-written student essays were excluded from the analysis. Despite linguistic differences, both ChatGPT and Bard proved to be highly proficient in generating essays reflecting students' personal experiences and viewpoints on real-life ethical dilemmas.
The Role of Large Language Models in Education
The introduction of LLMs like ChatGPT and Bard to the public domain has made AI tools more accessible, enabling the generation of human-like text based on user prompts. This accessibility has, however, raised concerns about potential misuse, particularly in educational settings.
Various organizations, including the European University Association and Open Universities Australia, have urged stakeholders to implement guidance and policies regarding the responsible use of AI in education. While some institutions have provided guidelines for students on the ethical use of AI tools like ChatGPT, a significant number of universities are yet to establish explicit policies concerning the application of AI writing tools.
Challenges and Considerations
Concerns about academic integrity violations have emerged, especially in official examinations where AI capabilities have been demonstrated in passing law and medical exams. The use of LLMs such as ChatGPT in essay writing poses challenges, particularly in detecting manual edits made by students post-generation. A survey conducted on students in Germany highlighted the widespread use of AI-based tools in studies, indicating the growing importance of AI-generated content in academic settings.
While students generally show positive attitudes towards AI in essay writing, concerns remain regarding its impact on creativity, critical thinking, and ethical considerations. As educational institutions continue to grapple with the implications of AI tools like ChatGPT, the development of clear policies and guidelines will be crucial to ensure their ethical and responsible use.
