Options
Towards a Trust Reliance Paradox? Exploring the Gap Between Perceived Trust in and Reliance on Algorithmic Advice
Journal
International Conference on Information Systems (ICIS)
Type
conference paper
Date Issued
2021
Research Team
IWI6
Abstract
Beyond AI-based systems’ potential to augment decision-making, reduce organizational resources, and counter human biases, unintended consequences of such systems have been largely neglected so far. Researchers are undecided on whether erroneous advice acts as an impediment to system use or is blindly relied upon. As part of an experimental study, we turn towards the impact of incorrect system advice and how to design for failure-prone AI. In an experiment with 156 subjects we find that, although incorrect algorithmic advice is trusted less, users adapt their answers to a system’s incorrect recommendations. While transparency on a system’s accuracy levels fosters trust and reliance in the context of incorrect advice, an opposite effect is found for users exposed to correct advice. Our findings point towards a paradoxical gap between stated trust and actual behavior. Furthermore, transparency mechanisms should be deployed with caution as their effectiveness is intertwined with system performance.
Language
English
Keywords
Decision-making
advice_taking
artificial_intelligence
transparency
trust
HSG Classification
contribution to scientific community
Publisher place
Austin, Texas
Event Title
International Conference on Information Systems (ICIS)
Event Location
Austin, Texas
Event Date
12.12.2021 - 15.12.2021
Subject(s)
Division(s)
Eprints ID
264499
File(s)