ChatGPT’s Impact on Students’ Performance in Solving Tasks Requiring Critical Thinking
Journal
Communications in Computer and Information Science (CCIS)
ISSN
978-3-031-99264-3
Type
conference paper
Date Issued
2025
Author(s)
Abstract
This study investigates the impact of ChatGPT-assistance compared to web search with Google on undergraduate students’ task and learning performance in solving tasks that require critical thinking. A supervised online experiment with 39 undergraduate students from a Swiss business university was conducted. Participants completed 11 critical thinking tasks, deliberately designed to prevent ChatGPT from generating complete solutions in response to a single prompt, and a post-knowledge test to assess potential learning gains. The experimental group (EG, n = 20) used the generative AI tool ChatGPT-4.0 for assistance, while the control group (CG, n = 19) relied on conventional web search with Google. Welch’s t-tests revealed no significant differences between groups in rubric-rated task performance (p = .71) or post-test scores (p = .93). Nevertheless, performance in analytical tasks approached statistical significance (p = .06), while surprisingly moderately favoring the Google-group (d = 0.63). These results challenge the assumption that AI tools like ChatGPT inherently enhance task performance. Instead, they indicate that independent critical thinking remains essential for achieving performance gains in complex tasks. The findings highlight the need for higher education curricula and task designs that actively integrate AI tools while preserving students’ cognitive engagement, ensuring that they serve as a complement rather than a substitute for critical thinking.
Language
English
Keywords
Generative AI
ChatGPT
Critical Thinking
Student Performance
Book title
Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium, Blue Sky, and WideAIED
Publisher
Springer Nature Switzerland
Publisher place
Cham
Volume
2591
Start page
218
End page
225
Event Title
AIED 2025
Subject(s)
Division(s)