Sanjiv Subodhnarayan Jha
Last Name
Jha
First name
Sanjiv Subodhnarayan
Email
sanjivsubodhnarayan.jha@unisg.ch
ORCID
Phone
+41 71 224 79 16
10 results
Now showing 1 - 10 of 10
- Some of the metrics are blocked by yourconsent settings
Item type:Publication, Temporal Scene Understanding using Contextually Unique Identification(2024-10-28); ; ; Scopus© Citations 5 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Temporal Scene Understanding using Contextually Unique IdentificationHumans can easily comprehend and explain the dynamics of a scene by observing the evolution of relationships among identified objects over time-while research towards Hybrid Intelligence promotes the integration of human and machine capabilities, this ability is currently beyond the capabilities of automated systems. Toward realizing it in automated scene understanding systems, we present interpretable Object Identification using Contextual Information System (OIC). Given a video, OIC detects, identifies, and tracks objects and their relationships over time to answer questions about the analyzed scene. Moreover, OIC makes predictions and infers the actors' intentions in a scene. To achieve this, our approach generates a scene graph containing classified objects and their semantic relationships. It then computes a Frame Graph by adding Contextually Unique IDentifiers (CUIDs) to each of the detected objects in the scene graph; the CUIDs permit tracking multiple object instances over time, even if the objects are visually identical. The CUIDs are then used to connect objects across a sequence of Frame Graphs, generating a Temporal Graph. This graph is exported as a cue for a pretrained Large Language Model to provide assistance and answer user questions. OIC's modular architecture enables simple comprehension and swapping of its components, making OIC more interpretable and maintainable than end-to-end scene understanding systems. Our quantitative and qualitative evaluation results demonstrate the effectiveness of OIC as a viable next step toward interpretable automated scene understanding systems.Type:conference paperJournal:2024 IEEE 36th International Conference on Tools with Artificial Intelligence (ICTAI)Scopus© Citations 5 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, The Spectrum of Proactive Functioning in Digital CompanionsThe future of proactive Digital Companions (DCs)—smart agents capable of assisting and protecting their users—lies in their ability to collaborate effectively with users, learn their preferences, and adjust their behavior according to the user’s current state and their environment. To achieve this, DCs must carefully strike a balance between acting autonomously, while keeping users informed to minimize inconveniences, thereby enhancing user acceptance. In this article, we conduct a user survey to enrich the architecture of proactive personal DCs that explores the trade-off between full autonomous functioning and ensuring sufficient user control in various critical and non-critical scenarios. Our findings indicate that most of the participants lean towards having control over the actions of DCs, and will actively collaborate with those systems in everyday situations, in which decisions are not urgent. However, participants would not mind yielding their control to DCs in time-sensitive or urgent scenarios. Furthermore, in our survey, participants highlighted the importance of explaining such actions well. Given the results from our survey, we implemented a system prototype with the enriched architecture of an explainable and unobtrusive proactive DC for a smart home environment. The actions of this DC are not always fully autonomous and follow our survey findings.Type:conference paperScopus© Citations 2 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, The Spectrum of Proactive Functioning in Digital CompanionsThe future of proactive Digital Companions (DCs)-smart agents capable of assisting and protecting their users-lies in their ability to collaborate effectively with users, learn their preferences, and adjust their behavior according to the user's current state and their environment. To achieve this, DCs must carefully strike a balance between acting autonomously, while keeping users informed to minimize inconveniences, thereby enhancing user acceptance. In this article, we conduct a user survey to enrich the architecture of proactive personal DCs that explores the trade-off between full autonomous functioning and ensuring sufficient user control in various critical and non-critical scenarios. Our findings indicate that most of the participants lean towards having control over the actions of DCs, and will actively collaborate with those systems in everyday situations, in which decisions are not urgent. However, participants would not mind yielding their control to DCs in time-sensitive or urgent scenarios. Furthermore, in our survey, participants highlighted the importance of explaining such actions well. Given the results from our survey, we implemented a system prototype with the enriched architecture of an explainable and unobtrusive proactive DC for a smart home environment. The actions of this DC are not always fully autonomous and follow our survey findings.Type:conference paperJournal:Proceedings of the International Conference on Mobile and Ubiquitous MultimediaScopus© Citations 2 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Actionable Contextual Explanations for Cyber-Physical SystemsOver the past two decades, Cyber-Physical Systems (CPS) have emerged as critical components in various industries, integrating digital and physical elements to improve efficiency and automation, from smart manufacturing and autonomous vehicles to advanced healthcare devices. However, the increasing complexity of CPS and their deployment in highly dynamic contexts undermine user trust. This motivates the investigation of methods capable of generating explanations about the behavior of CPS. To this end, Explainable Artificial Intelligence (XAI) methodologies show potential. However, these approaches do not consider contextual variables that a CPS may be subjected to (e.g., temperature, humidity), and the provided explanations are typically not actionable. In this article, we propose an Actionable Contextual Explanation System (ACES) that considers such contextual influences. Based on a user query about a behavioral attribute of a CPS (for example, vibrations and speed), ACES creates contextual explanations for the behavior of such a CPS considering its context. To generate contextual explanations, ACES uses a context model to discover sensors and actuators in the physical environment of a CPS and obtains time-series data from these devices. It then cross-correlates these time-series logs with the user-specified behavioral attribute of the CPS. Finally, ACES employs a counterfactual explanation method and takes user feedback to identify causal relationships between the contextual variables and the behavior of the CPS. We demonstrate our approach with a synthetic use case; the favorable results obtained, motivate the future deployment of ACES in real-world scenarios.Type:conference paperScopus© Citations 1 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, An Overview on the Explainability of Cyber-Physical SystemsType:conference paperJournal:Vol. 35 (2022): Proceedings of FLAIRS-35Scopus© Citations 8 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Scopus© Citations 2 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Poster: Towards Explaining the Effects of Contextual Influences
on Cyber-Physical SystemsType:conference posterJournal:11th International Conference on the Internet of Things (IoT ’21)Scopus© Citations 2 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, - Some of the metrics are blocked by yourconsent settings
Item type:Publication,