Who's the Tougher Grader: ChatGPT or Teachers? Quiz Yourself to...
Grading “assistants” that use generative artificial intelligence offer teachers a tantalizing promise: You can spend less time poring over hundreds of student essays while still giving kids the thoughtful feedback they need to make their writing better. But how do these tools actually stack up against experienced educators? Education Week wanted to find out.
Teacher Evaluation Experiment
We asked two teachers to grade the same piece of writing—a short persuasive essay from a 7th grader in Missouri—against a set of criteria for organization, clarity, and mechanics. Then, we gave the same instructions to ChatGPT, the generative AI chatbot developed by OpenAI.
The teachers involved have both used AI tools to help them give feedback on student work in their classes. Even so, both emphasize that they’re always involved in the process, reviewing AI’s comments or making the final decision about how to score a piece of writing.
Writing Sample Analysis
The prompt—“Should people ever wear pajamas in public? Why?”—asks students to defend their position on this issue as a formative measure of their ability to take a stance and support it with evidence. Students were directed to make a claim, provide reasons and evidence, offer a counterexample, and include a conclusion. They received the set of criteria that the teacher would use to evaluate their papers and some general guidance on writing for organization and clarity.
The student writing sample provided in this experiment showcased arguments against wearing pajamas in public, citing reasons such as lack of presentability, original purpose of pajamas, and availability of alternative comfortable outfits for public wear.
Responses and Quiz
Education Week provided all this information to the teachers and to ChatGPT. All three responses are outlined below. Take the quiz to see if you can identify whose response belongs to whom—and whether AI can match teachers’ insights.
Pajamas should never be worn in public. First of all, they are not a good representation that you would want to be seen as in public. Pajamas state a message that you didn't put in effort to make yourself presentable or simply just don't care. Pajamas are seen by many as something that should only be worn to sleep in, so wearing them in public may have a negative affect on your reputation.
Second of all, pajamas are not made to be worn out in public anyways. They are made to be slept in and not to be worn out of the house. If they are not used for their original purpose then there isn't a point in having them. If all you have is pajamas in your wardrobe then you will not have anything to wear to particularly serious events, like a nice dinner or a program.
Some may say that pajamas are acceptable in public because they are comfortable and relaxing, however there are brands that make comfortable outfits that are still more presentable to be worn out of the house, such as Lululemon and Vuori.
In conclusion, pajamas should never be worn in public because they do not give off a positive representation of yourself and are not made for that purpose but to specifically be worn to bed.
Scroll down to see Rubrics A, B, and C. Click the tabs to view each graded rubric.
Coverage of education technology is supported in part by a grant from the Siegel Family Endowment, at www.siegelendowment.org. Education Week retains sole editorial control over the content of this coverage.
