An Expert Digital Companion for Working Environments

Research in proactive agents capable of anticipating users’ needs has been conducted in different application areas, from agents aiming to help users accomplish their next task while using a text editor, to digital assistants that provide support to medical professionals in hospitals. Considering these works and following the rapid development in machine learning, the growing adoption of the IoT, the prevalence of pervasive computing, and the various innovative user interfaces that are becoming mainstream, we propose the creation of expert digital companions. Expert digital companions are agents that know the specifics of an environment, the available physical equipment in a space, as well as software components that might be reachable in such an environment. These expert digital companions can interact with personal digital companions, which know the preferences of a specific user. Thus, expert and personal digital companions collaborate to provide customised assistance to a user given her environment. In this paper, we present an implementation of an expert digital companion for employees working in an office and a shop-floor environment that takes advantages of: the decoupling that the W3C WoT Thing Description provides to interact with and control devices and other tools; a Knowledge Graph to provide richer descriptions of elements in the environment; a Computer Vision Algorithm to perceive the physical world; Mixed Reality as the medium to deliver assistance to workers.