Video Anomaly Detection Via Predictive Autoencoder with Gradient-Based Attention

Video anomaly detection is a challenging problem due to the ambiguity and diversity of anomalies in different scenes. In this paper, we present a novel framework to detect abnormal in surveillance videos. Inspired by the common deep reconstruction methods and deep prediction ones, we propose a new two-branch predictive autoencoder, including a reconstruction decoder and a prediction decoder, in which the prediction decoder is used to generate future frame and carry out anomaly detection by comparing the difference between predicted future frame and its ground truth. And the reconstruction decoder reconstructs the current frame, which can constrains the encoder to learn video representations better. Moreover, reconstruction decoder provides a gradient-based attention, which significantly helps the prediction decoder to generate higher quality future frame. Our method unifies reconstruction and prediction methods in an end-to-end framework, and it obtains impressive results with better predicted future frame on some publicly available datasets including CUHK Avenue and UCSD Pedestrian.