LatentEditor: Text Driven Local Editing of 3D Scenes
CoRR(2023)
摘要
While neural fields have made significant strides in view synthesis and scene
reconstruction, editing them poses a formidable challenge due to their implicit
encoding of geometry and texture information from multi-view inputs. In this
paper, we introduce \textsc{LatentEditor}, an innovative framework designed to
empower users with the ability to perform precise and locally controlled
editing of neural fields using text prompts. Leveraging denoising diffusion
models, we successfully embed real-world scenes into the latent space,
resulting in a faster and more adaptable NeRF backbone for editing compared to
traditional methods. To enhance editing precision, we introduce a delta score
to calculate the 2D mask in the latent space that serves as a guide for local
modifications while preserving irrelevant regions. Our novel pixel-level
scoring approach harnesses the power of InstructPix2Pix (IP2P) to discern the
disparity between IP2P conditional and unconditional noise predictions in the
latent space. The edited latents conditioned on the 2D masks are then
iteratively updated in the training set to achieve 3D local editing. Our
approach achieves faster editing speeds and superior output quality compared to
existing 3D editing models, bridging the gap between textual instructions and
high-quality 3D scene editing in latent space. We show the superiority of our
approach on four benchmark 3D datasets, LLFF, IN2N, NeRFStudio and NeRF-Art.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要