An attention-based deep learning approach for inertial motion recognition and estimation in human-robot collaboration

In line with a human-centric smart manufacturing vision, human-robot collaboration is striving to combine robots’ high efficiency and quality with humans’ rapid adaptability and high flexibility. In particular, perception, recognition and estimation of human motion determine when and what robot to collaborate with humans. This work presents an attention-based deep learning approach for inertial motion recognition and estimation in order to infer when robotic assistance will be requested by the human and to allow the robot to perform partial human tasks. First, in the stage of motion perception and recognition, quaternion-based calibration and forward kinematic analysis methods enable the reconstruction of human motion based on data streaming from an inertial motion capture device. Then, in the stage of motion estimation, residual module and Bidirectional Long Short-Term Memory module are integrated with proposed attention mechanism for estimating arm motion trajectories further. Experimental results show the effectiveness of the proposed approach in achieving better recognition and estimation in comparison with traditional approaches and existing deep learning approaches. It is experimentally verified in a laboratory environment involving a collaborative robot employed in a small part assembly task.