Human Intention Estimation Using Fusion of Pupil and Hand Motion

This paper addresses the problem of human intention inference in the context of human-robot collaboration by fusing information from both hand motion obtained using skeletal tracking and eye gaze obtained using pupil tracking to predict a human’s current intention. Intention is modeled as a motion profile that converges to a goal location. A Kalman filter is used on eye gaze data to obtain gaze point estimates. The gaze estimates are transformed into a reference frame common to the hand data. An IMM filter that tracks hand motion is designed which takes advantage of the gaze filter’s model probabilities by fusing them with its own. The fusion is performed with user chosen parameters that determine the degree to which each filter’s predictions are weighed over time. An experiment is designed to show the utility of the proposed algorithm in a setting in which multiple reaching tasks are completed in an unknown order. The results show that the proposed algorithm can accurately predict the human’s intention before the tasks are completed.