Exploring Perceptual and Motor Gestalt in Touchless Interactions with Distant Displays.

ABSTRACTMarkerless motion-sensing promises to position touchless interactions successfully in various domains (e.g., entertainment or surgery) because they are deemed natural. This naturalness, however, depends upon the mechanics of touchless interaction that remains largely unexplored. My dissertation first aims to deconstruct the interaction mechanics of touchless, especially its device-less property, from an embodied perspective. Grounded in this analysis, I then plan to investigate how visual perception affects touchless interaction with distant, 2D displays. Preliminary findings suggest that Gestalt principles in visual perception and motor action affect the touchless user experience. User interface elements demonstrating perceptual-grouping principles, such as similarity of orientation decreased users' efficiency, while continuity of UI elements forming a perceptual whole increased users' effectiveness. Moreover, following the law of Prägnanz, users often gestured to minimize their energy expenditure. This work can inform the design of touchless UX by uncovering relations between perceptual and motor gestalt in touchless interactions.