ChatGPT’s Impact on Students’ Performance in Solving Tasks Requiring Critical Thinking

This study investigates the impact of ChatGPT-assistance compared to web search with Google on undergraduate students’ task and learning performance in solving tasks that require critical thinking. A supervised online experiment with 39 undergraduate students from a Swiss business university was conducted. Participants completed 11 critical thinking tasks, deliberately designed to prevent ChatGPT from generating complete solutions in response to a single prompt, and a post-knowledge test to assess potential learning gains. The experimental group (EG, n = 20) used the generative AI tool ChatGPT-4.0 for assistance, while the control group (CG, n = 19) relied on conventional web search with Google. Welch’s t-tests revealed no significant differences between groups in rubric-rated task performance (p = .71) or post-test scores (p = .93). Nevertheless, performance in analytical tasks approached statistical significance (p = .06), while surprisingly moderately favoring the Google-group (d = 0.63). These results challenge the assumption that AI tools like ChatGPT inherently enhance task performance. Instead, they indicate that independent critical thinking remains essential for achieving performance gains in complex tasks. The findings highlight the need for higher education curricula and task designs that actively integrate AI tools while preserving students’ cognitive engagement, ensuring that they serve as a complement rather than a substitute for critical thinking.