RoHM: Robust Human Motion Reconstruction via Diffusion
CoRR(2024)
摘要
We propose RoHM, an approach for robust 3D human motion reconstruction from
monocular RGB(-D) videos in the presence of noise and occlusions. Most previous
approaches either train neural networks to directly regress motion in 3D or
learn data-driven motion priors and combine them with optimization at test
time. The former do not recover globally coherent motion and fail under
occlusions; the latter are time-consuming, prone to local minima, and require
manual tuning. To overcome these shortcomings, we exploit the iterative,
denoising nature of diffusion models. RoHM is a novel diffusion-based motion
model that, conditioned on noisy and occluded input data, reconstructs
complete, plausible motions in consistent global coordinates. Given the
complexity of the problem – requiring one to address different tasks
(denoising and infilling) in different solution spaces (local and global
motion) – we decompose it into two sub-tasks and learn two models, one for
global trajectory and one for local motion. To capture the correlations between
the two, we then introduce a novel conditioning module, combining it with an
iterative inference scheme. We apply RoHM to a variety of tasks – from motion
reconstruction and denoising to spatial and temporal infilling. Extensive
experiments on three popular datasets show that our method outperforms
state-of-the-art approaches qualitatively and quantitatively, while being
faster at test time. The code will be available at
https://sanweiliti.github.io/ROHM/ROHM.html.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要