A Conversational Agent to Improve Response Quality in Course Evaluations

Item Type Conference or Workshop Item (Paper)
Abstract Recent advances in Natural Language Processing (NLP) bear the opportunity to design new forms of human-computer interaction with conversational interfaces. We hypothesize that these interfaces can interactively engage students to increase response quality of course evaluations in education compared to the common standard of web surveys. Past research indicates that web surveys come with disadvantages, such as poor response quality caused by inattention, survey fatigue or satisficing behavior. To test if conversational interfaces have a positive impact on the level of enjoyment and the response quality, we design an NLP-based conversational agent and deploy it in a field experiment with 127 students in our lecture and compare it with a web survey as a baseline. Our findings indicate that using conversational agents for evaluations are resulting in higher levels of response quality and level of enjoyment, and are therefore, a promising approach to increase the effectiveness of surveys in general.
Authors Wambsganss, Thiemo; Winkler, Rainer; Söllner, Matthias & Leimeister, Jan Marco
Journal or Publication Title Conference on Human Factors in Computing Systems (CHI)
Language English
Subjects computer science
information management
education
HSG Classification contribution to scientific community
HSG Profile Area SoM - Business Innovation
Refereed Yes
Date April 2020
Event Title Conference on Human Factors in Computing Systems (CHI)
Event Location Honolulu, Hawaii
Event Dates 25-30 April 2020
Publisher DOI https://doi.org/10.1145/3334480.3382805
Official URL https://doi.org/10.1145/3334480.3382805
Depositing User Thiemo Wambsganss
Date Deposited 28 Feb 2020 16:55
Last Modified 20 Jul 2022 17:41
URI: https://www.alexandria.unisg.ch/publications/259507

Download

[img] Text
LBW349.pdf - Published Version
Download (1MB)

Citation

Wambsganss, Thiemo; Winkler, Rainer; Söllner, Matthias & Leimeister, Jan Marco: A Conversational Agent to Improve Response Quality in Course Evaluations. 2020. - Conference on Human Factors in Computing Systems (CHI). - Honolulu, Hawaii.

Statistics

https://www.alexandria.unisg.ch/id/eprint/259507
Edit item Edit item
Feedback?