Coupling Deep Discriminative And Generative Models For Reactive Robot Planning In Human-Robot Collaboration

Human-robot collaboration towards achieving a common goal is most effective when the robot has the capability to estimate the intentions and needs of its human partner, and to plan complementary actions accordingly. To this end, synergistic coupling between inference engines and task-planning algorithms is essential: the earlier the robot can anticipate the actions performed by its partner, the safer and more seamless the interaction between the two parties will be.In this work, we propose a perception-based analytics framework that incorporates discriminative and generative models, which together estimate the current and future class of an action being performed by a human. The analytics leverage a sequence of human skeletal joint locations extracted from a depth map video stream of the human partner. The generative model ingests current and previous joint positions and outputs a sequence of predicted future positions. The discriminative model produces a vector of probabilities indicating the likelihood that the future action belongs to each class within a set of action classes being considered. The information on current and future actions is fed to a task planning module which selects the robot collaborative action that better suits the estimated present and future human states.