Man vs Machine: Can AI Grade and Give Feedback Like a Human? - Arnaud Chevalier; Jakub Orzech; Petar Stankov (Institute of Labor Economics (IZA), 2024)
Grading and providing feedback are two of the most time-consuming activities in education. We developed a randomised controlled trial (RCT) to test whether they could be performed by generative artificial intelligence (Gen-AI). We randomly allocated undergraduate students to feedback provided either by a human instructor, ChatGPT 3.5, or ChatGPT 4. Our results show that: (i) Students treated with the freely accessible ChatGPT 3.5 received lower grades in subsequent assessments than their peers in the control group who always received human feedback; (ii) No such penalty was observed for ChatGPT 4. Separately, we tested the capacity of Gen-AI to grade student work. Gen-AI grades and ranks were significantly different than human-generated grades. Overall, while the newest LLM helps learning as well as a human, its ability to grade student work is still inferior.
Keywords: A22; C93; I23; I24; 330; feeback; grading; Artificial Intelligence; learning with Gen-AI