Using Multimodal Sensing to Improve Awareness in Human-AI Interaction

Abstract

In recent years, we have leveraged a broad range of AI techniques to improve our understanding and use of implicit human inputs for enhancing the capabilities of future AI-infused systems. At the same time, these new capabilities have given rise to novel interactions with AI, which require HCI techniques for improving its use through design and evaluation. In this paper, we promote the use of AI-supported unobtrusive multimodal sensing by presenting two ongoing projects that together explore intention, attention and activity recognition for developing and enabling three facets of awareness respectively—situation-, cognition- and context-awareness. Our collective efforts show a snapshot of ho..