Multi-scale 2D Temporal Map Diffusion Models for Natural Language Video Localization
CoRR(2024)
摘要
Natural Language Video Localization (NLVL), grounding phrases from natural
language descriptions to corresponding video segments, is a complex yet
critical task in video understanding. Despite ongoing advancements, many
existing solutions lack the capability to globally capture temporal dynamics of
the video data. In this study, we present a novel approach to NLVL that aims to
address this issue. Our method involves the direct generation of a global 2D
temporal map via a conditional denoising diffusion process, based on the input
video and language query. The main challenges are the inherent sparsity and
discontinuity of a 2D temporal map in devising the diffusion decoder. To
address these challenges, we introduce a multi-scale technique and develop an
innovative diffusion decoder. Our approach effectively encapsulates the
interaction between the query and video data across various time scales.
Experiments on the Charades and DiDeMo datasets underscore the potency of our
design.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要