Conference Proceedings

Using Multimodal Sensing to Improve Awareness in Human-AI Interaction

Joshua Newn

Proceedings of the CHI 2020 Workshop on Artificial Intelligence for HCI: A Modern Approach | Association for Computing Machinery (ACM) | Published : 2020


In recent years, we have leveraged a broad range of AI techniques to improve our understanding and use of implicit human inputs for enhancing the capabilities of future AI-infused systems. At the same time, these new capabilities have given rise to novel interactions with AI, which require HCI techniques for improving its use through design and evaluation. In this paper, we promote the use of AI-supported unobtrusive multimodal sensing by presenting two ongoing projects that together explore intention, attention and activity recognition for developing and enabling three facets of awareness respectively—situation-, cognition- and context-awareness. Our collective efforts show a snapshot of ho..

View full abstract