The Generative AI Learning Penalty: Evidence from Chinese Secondary Education
Using 30 months of panel data on 26,811 Chinese students in grades 7-12, we study how generative AI affects homework productivity and learning. The data combine monthly closed-book exams, high-school and college entrance exams, and homework scores and completion time across nine subjects. We exploit staggered AI adoption in a difference-in-differences design. AI adoption raises homework scores by 18% and reduces completion time by 30%, but lowers monthly exam scores by 20% within six months. High-stakes entrance-exam scores fall by 18 and 24%, with the full penalty emerging only after about two years. The losses are largest in social science subjects, followed by STEM and languages, and are especially large for junior students, high-achieving students, and boys. The learning losses are concentrated among roughly 80% of AI users whose behavior is consistent with homework outsourcing, as indicated by exceptionally short homework completion time coupled with high homework scores. AI users who maintain similar homework completion time as non-AI users experience small learning losses.
Edit: moving my comment up here
Just in case as it’s formatted a bit weirdly
X-Axis: Homework scores
Y-Axis: Exam scores
The implication being that if you use AI to do your homework and score highly on it, when it comes time to take the exam you'll do very badly. If you don't use AI to do your homework, your exam scores will more or less match your homework scores.
Still, the shapes of the curves are weird. I guess it suggests that students scoring around 100 on their homework are probably not relying on AI overly much, so their exam scores are still decently high. But, every student who has a higher homework score is relying on AI and, as a result, their exams suffer?
What really sucks about this is that I don't think students have the self-discipline to not use AI. I'm old enough now to know that cutting corners to get good grades is dumb if it means I'm not learning. But, I wasn't smart enough to know that back then. I absolutely would have cut corners to get those good grades and get through the homework more quickly. My parents wouldn't have approved, but they weren't technologically sophisticated enough to stop me.