Hands-Free VR
CoRR(2024)
摘要
The paper introduces Hands-Free VR, a voice-based natural-language interface
for VR. The user gives a command using their voice, the speech audio data is
converted to text using a speech-to-text deep learning model that is fine-tuned
for robustness to word phonetic similarity and to spoken English accents, and
the text is mapped to an executable VR command using a large language model
that is robust to natural language diversity. Hands-Free VR was evaluated in a
controlled within-subjects study (N = 22) that asked participants to find
specific objects and to place them in various configurations. In the control
condition participants used a conventional VR user interface to grab, carry,
and position the objects using the handheld controllers. In the experimental
condition participants used Hands-Free VR. The results confirm that: (1)
Hands-Free VR is robust to spoken English accents, as for 20 of our
participants English was not their first language, and to word phonetic
similarity, correctly transcribing the voice command 96.71
Hands-Free VR is robust to natural language diversity, correctly mapping the
transcribed command to an executable command in 97.83
Hands-Free VR had a significant efficiency advantage over the conventional VR
interface in terms of task completion time, total viewpoint translation, total
view direction rotation, and total left and right hand translations; (4)
Hands-Free VR received high user preference ratings in terms of ease of use,
intuitiveness, ergonomics, reliability, and desirability.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要