Jan Marco Leimeister

Conversational agents (CAs) provide opportunities for improving the interaction in evaluation surveys. To investigate if and how a user-centered conversational evaluation tool impacts users' response quality and their experience, we build EVA - a novel conversational course evaluation tool for educational scenarios. In a field experiment with 128 students, we compared EVA against a static web survey. Our results confirm prior findings from literature about the positive effect of conversational evaluation tools in the domain of education. Second, we then investigate the differences between a voice-based and text-based conversational human-computer interaction of EVA in the same experimental set-up. Against our prior expectation, the students of the voice-based interaction answered with higher information quality but with lower quantity of information compared to the text-based modality. Our findings indicate that using a conversational CA (voice and text-based) results in a higher response quality and user experience compared to a static web survey interface.

We present an annotation approach to capturing emotional and cognitive empathy in student-written peer reviews on business models in German. We propose an annotation scheme that allows us to model emotional and cognitive empathy scores based on three types of review components. Also, we conducted an annotation study with three annotators based on 92 student essays to evaluate our annotation scheme. The obtained inter-rater agreement of α = 0.79 for the components and the π = 0.41 for the empathy scores indicate that the proposed annotation scheme successfully guides annotators to a substantial to moderate agreement. Moreover, we trained predictive models to detect the annotated empathy structures and embedded them in an adaptive writing support system for students to receive individual empathy feedback independent of an instructor, time, and location. We evaluated our tool in a peer learning exercise with 58 students and found promising results for perceived empathy skill learning, perceived feedback accuracy, and intention to use. Finally, we present our freely available corpus of 500 empathy-annotated, student-written peer reviews on business models and our annotation guidelines to encourage future research on the design and development of empathy support systems.

Smart Personal Assistants (SPAs; such as Amazon’s Alexa or Google’s Assistant) let users interact with computers in a more natural and sophisticated way that was not possible before. Although there exists an increasing amount of research of SPA technology in education, empirical evidence of its ability to offer dynamic scaffolding to enhance students problem-solving skills is still scarce. To fill this gap, the aim of this paper is to find out whether interactions with scaffolding-based SPA technology enable students to internalize and apply problem-solving steps on their own in a 10th grade high school and a vocational business school class. Students in the experiment classes completed their assignments using Smart Personal Assistants, whereas students in the control classes completed the same assignments using traditional methods. This study used a mixed-method approach consisting of two field quasi-experiments and one post-experiment focus group discussion. The empirical results revealed that students in the experiment classes acquired significantly more problem-solving skills than those in the control classes (Study 1: p = 0.0396, study 2: p < 0.001), and also uncovered several changes in students’ learning processes. The findings provide first empirical evidence for the value of using SPA technology on skill development in general, and on problem-solving skill development in particular.

The success of innovative teaching/learning approaches aiming to foster problem solving in management education depends on useful and easy-to-use IT components in the learning process. However, the complexity of problem solving in self-regulated learning approaches may overwhelm the learner and can lead to unsatisfying learning outcomes. Research suggests the implementation of technology-enhanced scaffolds as a mechanism to guide the learners in their individual problem-solving process to enhance their learning outcomes. We present a theoretical model based on adaptive structuration theory and cognitive load theory that explains how technology-enhanced scaffolding contributes to learning outcomes. We test the model with a fully randomized between-subject experiment in a flipped classroom for management education focusing on individual problem solving. Our results show that technology-enhanced scaffolding contributes significantly to the management of cognitive load as well as to learning process satisfaction and problem-solving learning outcomes. Thereby, our paper provides new conceptual and empirically tested insights for a better understanding of technology-enhanced scaffolds and their design to assist problem solving and its respective effects in flipped classrooms for management education.

In the realm of smart services, smart personal assistants (SPAs) have become a popular medium for value co-creation between service providers and users. The market success of SPAs is largely based on their innovative material properties, such as natural language user interfaces, machine-learning-powered request handling and service provision, and anthropomorphism. In different combinations, these properties offer users entirely new ways to intuitively and interactively achieve their goals and, thus, co-create value with service providers. But how does the nature of the SPA shape value co-creation processes? In this paper, we look through a functional affordances lens to theorize about the effects of different types of SPAs (i.e., with different combinations of material properties) on users' value co-creation processes. Specifically, we collected SPAs from research and practice by reviewing scientific literature and web resources, developed a taxonomy of SPAs' material properties, and performed a cluster analysis to group SPAs of a similar nature. We then derived 2 general and 11 cluster-specific propositions on how different material properties of SPAs can yield different affordances for value co-creation. With our work, we point out that smart services require researchers and practitioners to fundamentally rethink value co-creation as well as revise affordances theory to address the dynamic nature of smart technology as a service counterpart.

This report summarizes the discussion in a panel session on gamification designs at the 2019 European Conference on Information Systems in Stockholm, Sweden. The panel explores a research agenda for gamification design. The “what, why, and how” are considered to analyze the current state of the art of gamification research. An adapted defini-tion of gamification is presented as one outcome of the workshop to better describe what gamification is and what it can be used for. “Why” and “how” to employ gamification are discussed for different contexts. This can be used to gamify information systems, identity outcomes that are addressed by gamification concepts, and explore new ways of how to gamify. Overall, the panel presents new areas for future research and practice by identifying innovative ways to bring existing gamification concepts to a more impactful level.

Many governments and organizations recognize the potential of open innovation (OI) models to create value with large numbers of people beyond the organization. It can be challenging , however, to design an effective collaborative process for a massive group. Collaboration engineering (CE) is an approach for the design and deployment of repeatable collaborative work practices that can be executed by practitioners themselves without the ongoing support of external collaboration engineers. To manage the complexity of the design process, they use a modeling technique called facilitation process models (FPM) to capture high-level design decisions that serve different purposes, such as documenting and communicating a design, etc. FPM, however, was developed to support designs for groups of fewer than 100 people. It does not yet represent design elements that become important when designing for groups of hundreds or thousands of participants, which can be found in many OI settings. We use a design science approach to identify the limitations of the original FPM and derive requirements for extending FPM. This article contributes to the CE and to the OI literature by offering an FPM 2.0 that assists CE designers to design new OI processes, with a special focus on outside-in OI.