Mitigating inherent latency in off-device computer vision for spatial interfaces

Researchers commonly need to deploy specialised computer vision models on Mixed Reality devices, such as Microsoft HoloLens, to extract richer contextual information. This paper introduces an approach to alleviate the negative impacts of elevated latency caused by model inference time, network transactions, and photo capture during the use of external deep learning models. In contrast to previous methods that constrained user movements, our approach augments users' environmental perception while preserving freedom of movement.