Options
Anuschka Schmitt
Last Name
Schmitt
First name
Anuschka
Email
anuschka.schmitt@unisg.ch
Phone
+41 71 224 3225
Now showing
1 - 5 of 5
-
PublicationConversational Agents for Information Retrieval in the Education Domain: A User-Centered Design Investigation( 2022-11-11)Text-based conversational agents (CAs) are widely deployed across a number of daily tasks, including information retrieval. However, most existing agents follow a default design that disregards user needs and preferences, ultimately leading to a lack of usage and an unsatisfying user experience. To better understand how CAs can be designed in order to lead to effective system use, we deduced relevant design requirements from both literature and 13 user interviews. We built and tested a question-answering, text-based CA for an information retrieval task in an education scenario. Results from our experimental test with 41 students indicate that following a user-centered design has a significant positive effect on enjoyment and trust in a CA as opposed to deploying a default CA. If not designed with the user in mind, CAs are not necessarily more beneficial than traditional question-answering systems. Beyond practical implications for effective CA design, this paper points towards key challenges and potential research avenues when deploying social cues for CAs.Type: journal articleJournal: Proceedings of the ACM on Human-Computer Interaction (PACMHCI)Volume: 6Issue: CSCW2
-
PublicationUnleashing Process Mining for Education: Designing an IT-Tool for Students to Self-Monitor their Personal Learning Paths( 2022-02-23)
;Oeste-Reiß, SarahThe ability of students to self-monitor their learning paths is in demand as never before due to the recent rise of online education formats, which entails less interaction with lecturers. Recent advantages in educational process mining (EPM) offer new opportunities to monitor students’ learning paths by processing log data captured by technology-mediated learning environments. However, current literature falls short on providing user-centered design principles for IT-tools which can monitor learning paths using EPM. Hence, in this paper, we examine how to design a self-monitoring tool that supports students to evaluate their learning paths. Based on theoretical insights of 66 papers and nine user interviews, we propose seven design principles for an IT-tool which facilitates self-monitoring for students based on EPM. Further, we evaluate the design principles with seven potential users. Our results demonstrate a promising approach to help students improve their self-efficacy in their individual learning process using EPM.Type: conference paperJournal: Internationale Tagung Wirtschaftsinformatik (WI) -
PublicationDesigning for Conversational System Trustworthiness: The Impact of Model Transparency on Trust and Task Performance( 2022-06-24)Designing for system trustworthiness promises to address challenges of opaqueness and uncertainty introduced through Machine Learning (ML)-based systems by allowing users to understand and interpret systems’ underlying working mechanisms. However, empirical exploration of trustworthiness measures and their effectiveness is scarce and inconclusive. We investigated how varying model confidence (70% versus 90%) and making confidence levels transparent to the user (explanatory statement versus no explanatory statement) may influence perceptions of trust and performance in an information retrieval task assisted by a conversational system. In a field experiment with 104 users, our findings indicate that neither model confidence nor transparency seem to impact trust in the conversational system. However, users’ task performance is positively influenced by both transparency and trust in the system. While this study considers the complex interplay of system trustworthiness, trust, and subsequent behavioral outcomes, our results call into question the relation between system trustworthiness and user trust.Type: conference paperJournal: European Conference on Information Systems (ECIS)
-
PublicationTowards a Trust Reliance Paradox? Exploring the Gap Between Perceived Trust in and Reliance on Algorithmic Advice( 2021)Beyond AI-based systems’ potential to augment decision-making, reduce organizational resources, and counter human biases, unintended consequences of such systems have been largely neglected so far. Researchers are undecided on whether erroneous advice acts as an impediment to system use or is blindly relied upon. As part of an experimental study, we turn towards the impact of incorrect system advice and how to design for failure-prone AI. In an experiment with 156 subjects we find that, although incorrect algorithmic advice is trusted less, users adapt their answers to a system’s incorrect recommendations. While transparency on a system’s accuracy levels fosters trust and reliance in the context of incorrect advice, an opposite effect is found for users exposed to correct advice. Our findings point towards a paradoxical gap between stated trust and actual behavior. Furthermore, transparency mechanisms should be deployed with caution as their effectiveness is intertwined with system performance.Type: conference paperJournal: International Conference on Information Systems (ICIS)
-
PublicationThe Potential of Technology-Mediated Learning Processes: A Taxonomy and Research Agenda for Educational Process Mining( 2021-12-15)
;Mahning, Thomas ;Ott, Anja ;Soellner, Sigita ;Ngo, Ngoc Anh ;Geyer-Klingeberg, Jerome ;Nakladal, JaninaType: conference paperJournal: International Conference on Information Systems (ICIS)