Complementary Tracking and Two-Handed Interaction for Remote 3 D Medical Consultation with a PDA

We report here on applying a complementary tracking and twohanded interaction paradigm to remote medical consultation using a mobile hand-held device. The basic idea is to track the relative pose between the mobile device and an integrated but detachable “patient surrogate” such as the device cover, enabling 6 DOF visualization of streaming vision-based 3D reconstructions of a remote patient. The two-handed device/target paradigm is complementary in that it enables 6 DOF user interaction with applications on the device, while making the tracking problem tractable.