ShadowDancXR: Body Gesture Digitization for Low-cost Extended Reality (XR) Headsets

Abstract

Low-cost, smartphone-based extended reality (XR) headsets, such as Google Cardboard, are more accessible but lack advanced features like body-tracking, which limits the expressiveness of interaction. We introduce ShadowDancXR, a technique that relies on the front-facing camera of the smartphone to reconstruct users' body gestures from projected shadows, which allows for wider range of interaction possibilities with low-cost XR. Our approach requires no embedded accessories and serves as an inexpensive, portable solution for body gesture digitization. We provide two interactive experiences to demonstrate the feasibility of our concept.