How Can AI Earn Trust of System Administrators in the IT-Security Domain?

As networks grow in size and complexity and the number of cyberattacks increases, securing the network becomes challenging. Partial workflow automation with the help of Artificial Intelligence (AI) has become a recent remedy to address these concerns and help system administrators to handle large volumes of data and keep the system safeguarded. However, although beneficial, AI-aided software solutions in the network security context often come at the cost of transparency and control, suffering from the opacity of their algorithms. The subsequent lack of understanding about the system, its decisions and outcomes cultivates mistrust on the system administrator’s behalf, which leads to the overall reluctance to use the new software solutions, slower response time in case of a security breach, and inability to resolve conflicts in the system. A recent study by Ehsan et al. focused on the concept of social transparency (ST). It showed that when applied to the IT-security context, peer support could help alleviate the issue of mistrust in the system and provide the missing knowledge to the sysadmin when faced with a novel situation. In this position paper, we outline our current in-progress work, where we investigate the goals and needs of our target group. We also test the applicability of the ST framework proposed by Ehsan et al. to the IT-security domain and its potential for facilitating greater trust in human-AI teams in system administration. Our first results show that ST can indeed yield benefits for sysadmins but only when coupled with other contextual information available in the system and only when it adheres to specific quality standards in terms of content it provides. As we continue our analysis, we aim to compile a set of desirable user requirements for a successful AI system adoption and support of human-AI teams in the network security domain.