Feature fusion for human action recognition based on classical descriptors and 3D convolutional networks

This paper proposes a feature fusion method that combines different kinds of classical descriptors and multi-channel 3-dimensional convolutional neural networks for the Human Action Recognition(HAR). The interrelationship between the classical descriptors and the 3D convolutional filters is explored. The spatio-temporal features are learned by the 3D convolutional networks which is trained on a large scale labeled video dataset. The classical descriptors are used as auxiliary feature to fuse a fusion feature vector with the learned features from 3D CNN. Feeding this new fusion feature vector into the SVM classifier can improve the recognition accuracy. The verification experiments are finished on different datasets. The recognition rate of the KTH dataset is 95.1% and that of the UCF101 dataset is 86.6% The experimental results prove that this feature fusion method performs efficient and robust on the human action recognition.