Study Reveals Artificial Intelligence May Not Enhance Student Learning

On September 11, Popular Science published an article titled "Children Using ChatGPT as a Learning Assistant Perform Worse on Exams," reported by Jill Bash. The article raises an important question: Does artificial intelligence truly aid student learning? Recent research conducted in a Turkish high school may provide some insights.

Researchers from the University of Pennsylvania found that high school students who used the conversational AI tool ChatGPT for practicing math problems scored worse on math exams compared to those who did not use it. While the students using ChatGPT solved 48% more practice problems, their test scores were 17% lower than their peers.

A third group of students used a more refined version of ChatGPT, designed to act more like a teaching assistant. This version offered hints without providing direct answers. Students using this AI assistant performed significantly better in practice sessions, answering 127% more questions correctly compared to those who practiced without technological aids. However, in a subsequent exam, these AI-assisted students did not outperform their peers who practiced independently; their scores were comparable.

In their paper titled "Does Generative AI Affect Learning?", the researchers warn parents and educators that freely available AI chatbots can significantly hinder learning. Even specialized versions designed to serve as teaching assistants may not provide the expected benefits. The researchers determined that the core issue lies in students using chatbots as "crutches." Their analysis revealed a tendency among students to seek simple answers, rather than developing their own problem-solving skills.

Additionally, the study noted that ChatGPT is prone to errors, correctly answering only about half of the math questions posed to it, and its step-by-step approach was wrong 42% of the time.