Template Free Reconstruction of Human-object Interaction with Procedural Interaction Generation
CoRR(2023)
摘要
Reconstructing human-object interaction in 3D from a single RGB image is a
challenging task and existing data driven methods do not generalize beyond the
objects present in the carefully curated 3D interaction datasets. Capturing
large-scale real data to learn strong interaction and 3D shape priors is very
expensive due to the combinatorial nature of human-object interactions. In this
paper, we propose ProciGen (Procedural interaction Generation), a method to
procedurally generate datasets with both, plausible interaction and diverse
object variation. We generate 1M+ human-object interaction pairs in 3D and
leverage this large-scale data to train our HDM (Hierarchical Diffusion Model),
a novel method to reconstruct interacting human and unseen objects, without any
templates. Our HDM is an image-conditioned diffusion model that learns both
realistic interaction and highly accurate human and object shapes. Experiments
show that our HDM trained with ProciGen significantly outperforms prior methods
that requires template meshes and that our dataset allows training methods with
strong generalization ability to unseen object instances. Our code and data
will be publicly released at:
https://virtualhumans.mpi-inf.mpg.de/procigen-hdm.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要