Cross-View Action Recognition Based on a Statistical Translation Framework

Actions captured under view changes pose serious challenges to modern action recognition methods. In this paper, we propose an effective approach for cross-view action recognition based on a statistical translation framework, which boils down to estimation of visual word transfer probabilities across views. Specifically, local features are extracted from action video frames and form bags of words based on k-means clustering. Though the appearance of an action may vary due to view changes, the underlying transfer tendency between visual words across views can be exploited. We propose two methods to measure the visual-word-based transfer relationship, which are eventually based on frequency counts of word pairs. In the first method, word transfer probabilities are estimated by maximizing the likelihood of a shared action set with the EM algorithm. In the second method, the word transfer probabilities are estimated by using likelihood-ratio tests. The two methods achieve comparable results and perform better when they are combined. For cross-view action classification, we compute action transfer probabilities based on the estimated word transfer probabilities, and then implement a K-NN-like classification based on action video transfer probabilities. We verified our method on the public multi-view IXMAS dataset and WVU dataset.