Play and Rewind: Context-aware Video Temporal Action Proposals

•We propose a novel context-aware temporal action proposal generation network for TAP task. It makes full use of the contextual information during both initial proposal generation and re-ranking phases.•For generating initial proposals, we design a Bi-directional Parallel LSTMs to extract the visual features of a video unit by considering its contextual information.•For re-ranking the proposals, we design an action-attention based re-ranking network which considers both surrounding proposal and initial actionness scores.•Extensive experiments on the challenging datasets demonstrate the effectiveness of our proposed framework.