Turn It This Way: Grounding Collaborative Action With Remote Gestures

Remote gesture systems have been shown to provide a significant enhancement to performance in collaborative physical tasks, an effect ascribed to the ability of remote gestures to help ground deictic references. The argument that this effect works by replacing complex referential descriptions with simple pointing behaviours has been drawn into question by recent research. In this paper we significantly unpack the effects of remote gesturing on collaborative language, arguing for a more complex role for remote gestures in interaction. We demonstrate how remote gestures influence the structure of collaborative discourse, and how their use can also influence the temporal nature of the grounding process. Through generating a deeper understanding of these effects of remote gesturing on collaborative language we derive implications for the development and deployment of these technologies.