Florian Mathis
Title
Dr.
Last Name
Mathis
First name
Florian
Email
florian.mathis@unisg.ch
Phone
+41 71 224 3876
12 results
Now showing 1 - 10 of 12
- Some of the metrics are blocked by yourconsent settings
Item type:Publication, LifeInsight: Design and Evaluation of an AI-Powered Assistive Wearable for Blind and Low Vision People Across Multiple Everyday Life ScenariosAssistive technologies (ATs) have the potential to empower blind and low vision (BLV) people. Yet, they often remain underutilised due to their immobility and limited applicability across scenarios. This paper presents LifeInsight, an AI-powered assistive wearable for BLV people that uses a wearable camera, microphone and single-click interface for goal-oriented visual querying. To inform the design of LifeInsight, we first collected a corpus of BLV people’s daily experiences using video probes and interviews. Ten BLV people recorded their daily experiences over one week using GoPro cameras, providing empirical insights. Based on these, we report on LifeInsight and its evaluation with 13 BLV people across six scenarios. LifeInsight effectively responded to visual queries, such as distinguishing between jars or identifying the status of a candle. Drawing on our work, we conclude with key lessons and practical recommendations to guide future research and advance the development and evaluation of AI-powered assistive wearables.Type:conference paperJournal:CHI Conference on Human Factors in Computing Systems (CHI ’25)Scopus© Citations 11 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, SpeculAR: An In-the-wild Exploration of Users’ Perceptions of a Social Augmented RealityWhile head-mounted augmented reality (AR) glasses are becoming increasingly reliable and relevant for public scenarios, little is known about users’ preferences and perceptions when experiencing AR in social spaces. In this late-breaking work, we investigate users’ perceptions of augmented social environments tailored to their shared interests. Through interviews with 64 participants, we first identified 12 key usage scenarios, which informed the development of SpeculAR, a prototype for an augmented social reality. We then conducted an in-the-wild study with 18 participants to assess their experiences with SpeculAR. Our findings reveal that shared AR scenarios are generally perceived positively and can be seamlessly integrated into real-world environments to augment and further enhance real-world spaces. Our exploration of augmented social spaces highlights important considerations and lays the groundwork for future studies and the realizations of augmented reality applications in everyday lives.Type:conference paperJournal:Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA ’25)Scopus© Citations 3 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, UrbAI: Exploring the Possibilities of Generative AI Image Processing to Promote Citizen Participation(2025-04-26); ; Giving citizens a voice in urban development processes is crucial for enabling socially sustainable cities and communities. However, citizens’ opportunities to express ideas are often limited to communication channels that offer poor incentives for participation. In this paper, we conducted an in-the-wild technology probe study (N=16) using a generative AI (GenAI) tool to allow citizens to visualise and submit urban development ideas by taking pictures and manipulating them with GenAI. The results highlight the potential of GenAI to empower, engage, and inspire citizens‘ creativity. We then conducted additional expert interviews (N=6) with city representatives and community associates. They voiced GenAI’s value in early-stage citizen participation but raised concerns about excluding senior citizens. Building on these insights, we present the design and evaluation (N=10) of UrbAI, a co-creative system tailored to urban development participation and conclude with lessons learned to inform how GenAI could be embedded in future citizen participation processes.Type:conference paperJournal:CHI Conference on Human Factors in Computing Systems (CHI ’25)Scopus© Citations 11 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, InterFACE: Establishing a Facial Action Unit Input Vocabulary for Hands-Free Extended Reality Interactions, From VR Gaming to AR Web Browsing(Association for Computing Machinery, 2025-04); Mcgill, MarkExtended Reality (XR) interactions often rely on spatial hand or controller inputs - necessitating dexterous wrist, hand and finger movements including pressing virtual buttons, pinching to select, and performing hand gestures. However, there are scenarios where such dependencies may render XR devices and apps inaccessible to users - from situational/temporary impairments such as encumbrance, to physical motor impairments. In this paper, we contribute to a growing literature considering facial input as an alternative. In a user study (N=20) we systematically evaluate the usability of 53 Facial Action Units in VR, deriving a set of optimal (comfort, effort, performance) FAUs for interaction. We then use these facial inputs to drive and evaluate (N=10) two demonstrator apps: VR locomotion, and AR web browsing, showcasing how close facial interaction can get to existing baselines, and demonstrating that FAUs offer a viable, generalizable input modality for XR devices.Type:conference paperJournal:CHI Conference on Human Factors in Computing Systems (CHI ’25)Scopus© Citations 2 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Everyday Life Challenges and Augmented Realities: Exploring Use Cases For, and User Perspectives on, an Augmented Everyday LifeAssistive technologies hold promise in empowering blind and low vision (BLV) people, assisting them in navigating the daily challenges they face. However, there remains limited understanding of the natu-ralistic everyday challenges experienced by BLV people, as well as how assistive technologies are specif-ically employed to effectively address these challenges. In this work, we use cultural probes to enhance our understanding of the challenges faced by BLV people and provide a corpus of insights into their eve-ryday life. We asked 10 BLV people to video record their naturalistic everyday life challenges for the du-ration of one week. These video probes, along with a set of semi-structured interviews, offer important empirical insights into the daily challenges. Despite the availability of assistive technologies and human support, where a BLV person receives assistance from another person, we observed a constant level of uncertainty where the probes revealed that BLV people often question an object's current state, such as the cleanliness of their clothing or the mix-ratio of multiple ingredients when preparing a meal. Fur-thermore, participants reported to experience significant delays in identifying critical observations, which leads to the situation where interventions may be too late to be effective (e.g., when a plant has a disease). By using video probes to provide insights into the daily challenges of BLV people, our corpus of naturalistic everyday life challenges offers a characterization of daily challenges faced by BLV people and informs future research efforts to co-design, co-develop, and co-evaluate novel assistive technologies that meet BLV people's preferences and their individual daily experiences.Type:conference paperJournal:AHs 2024Scopus© Citations 7 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, MRTranslate: Bridging Language Barriers in the Physical World Using a Mixed Reality Point-and-Translate SystemLanguage barriers pose significant challenges in our increasingly globalized world, hindering effective communication and access to information. Existing translation tools often disrupt the current activity flow and fail to provide seamless user experiences. In this paper, we contribute the design, implementation, and evaluation of \MRTranslate, an assistive Mixed Reality (MR) prototype that enables seamless translations of real-world text. We instructed 12 participants to translate items on a food menu using MRTranslate, which we compared to state-of-the-art translation apps, including Google Translate and Google Lens. Findings from our user study reveal that when utilising a fully functional implementation of MRTranslate, participants achieve success in up to 91.67% of their translations whilst also enjoying the visual translation of the unfamiliar text. Although the current translation apps were well perceived, participants particularly appreciated the convenience of not having to grab a smartphone and manually input the text for translation when using MRTranslate. We believe that MRTranslate, along with the empirical insights we have gained, presents a valuable step towards a future where MR transforms language translation and holds the potential to assist individuals in various day-to-day experiences.Type:conference paper - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Designing Grocery Shopping Experiences for Virtual RealityOnline grocery shopping offers time-saving efficiency and convenience, yet many people still prefer physical shopping for trust in food freshness and other sensory experiences. While online stores are evolving to offer new user experiences, such as supporting eco-friendly or ethical shopping, the desktop and mobile platforms limit the engagement of such experiences. Virtual Reality (VR) presents an opportunity to create immersive and rich grocery shopping experiences, closing the gap between the convenience of online shopping and the sensory experience of physical shopping. Yet, designing VR grocery stores remains relatively unexplored. In this paper, we investigate the long-term potential of VR grocery stores, focusing on meeting individual needs. Through a co-design workshop, participants brainstormed the design of product displays, in-shop navigation, shopping carts, social shopping, among others. Based on our findings, we provide design recommendations for future VR grocery shopping to develop meaningful alternatives to existing shopping experiences for groceries.Type:conference paperScopus© Citations 8 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Exploring Mobile Devices as Haptic Interfaces for Mixed Reality(ACM, 2024); Scopus© Citations 18 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Exploring User Perceptions on Visual CO2 Representations as Eco-Feedback in Virtual Reality(2025-04-26); ; ; Eco-feedback interventions primarily focus on lowering energy consumption behaviours instead of considering the impact of CO2 emissions resulting from the overall energy mix of renewable and non-renewable energy sources. Although labelling products with CO2 emission is an effective way for green nudging, consumers often perceive the information as intangible and incomprehensible. In this paper, we study the impact of the visual representation of CO2 equivalences (VisualRepCO2) on customers’ perceptions when choosing an energy supplier in virtual reality (VR) compared to a paper baseline and desktop setup. Our findings indicate that a VisualRepCO2 supports customers of electricity supplies in understanding their impact more comprehensibly and touches them more emotionally than a text label. We conclude by demonstrating lessons learned for future research and offering recommendations to support practitioners in enhancing customers’ understanding of CO2 emissions.Type:conference posterJournal:Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA ’25)Scopus© Citations 2 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, CitizenVision: A Mobile App to Promote Citizen Engagement with Generative AI(2025-07-05); ; Traditional methods for citizen participation in urban planning rely on written forms or hand-drawn sketches, which can be inaccessible to those without drawing or design skills. This limits the ability of citizens to communicate their visions effectively, leading to missed opportunities for community-driven improvements. We developed CitizenVision, a generative AI-powered app that enables citizens to visualize their ideas for public spaces by marking areas in photos and textual prompts. CitizenVision incorporates features such as map-based project discovery, keyword-based voting, and community idea browsing. A user study with N=19 participants comparing CitizenVision with a traditional paper-based method showed that CitizenVision significantly improves user experience and engagement. We demonstrate how generative AI can be effectively used in participatory urban planning, allowing citizens to express ideas creatively and interact with those of others. Our findings highlight potential and driving factors for developing similar tools and using them effectively for participatory urban design processes.Type:conference posterJournal:Designing Interactive Systems Conference (DIS ’25 Companion)Scopus© Citations 3