AnimatableDreamer: Text-Guided Non-rigid 3D Model Generation and Reconstruction with Canonical Score Distillation
arxiv(2023)
摘要
Advances in 3D generation have facilitated sequential 3D model generation
(a.k.a 4D generation), yet its application for animatable objects with large
motion remains scarce. Our work proposes AnimatableDreamer, a text-to-4D
generation framework capable of generating diverse categories of non-rigid
objects on skeletons extracted from a monocular video. At its core,
AnimatableDreamer is equipped with our novel optimization design dubbed
Canonical Score Distillation (CSD), which lifts 2D diffusion for temporal
consistent 4D generation. CSD, designed from a score gradient perspective,
generates a canonical model with warp-robustness across different
articulations. Notably, it also enhances the authenticity of bones and skinning
by integrating inductive priors from a diffusion model. Furthermore, with
multi-view distillation, CSD infers invisible regions, thereby improving the
fidelity of monocular non-rigid reconstruction. Extensive experiments
demonstrate the capability of our method in generating high-flexibility
text-guided 3D models from the monocular video, while also showing improved
reconstruction performance over existing non-rigid reconstruction methods.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要