DreamScene360: Unconstrained Text-to-3D Scene Generation with Panoramic Gaussian Splatting
arxiv(2024)
摘要
The increasing demand for virtual reality applications has highlighted the
significance of crafting immersive 3D assets. We present a text-to-3D
360^∘ scene generation pipeline that facilitates the creation of
comprehensive 360^∘ scenes for in-the-wild environments in a matter of
minutes. Our approach utilizes the generative power of a 2D diffusion model and
prompt self-refinement to create a high-quality and globally coherent panoramic
image. This image acts as a preliminary "flat" (2D) scene representation.
Subsequently, it is lifted into 3D Gaussians, employing splatting techniques to
enable real-time exploration. To produce consistent 3D geometry, our pipeline
constructs a spatially coherent structure by aligning the 2D monocular depth
into a globally optimized point cloud. This point cloud serves as the initial
state for the centroids of 3D Gaussians. In order to address invisible issues
inherent in single-view inputs, we impose semantic and geometric constraints on
both synthesized and input camera views as regularizations. These guide the
optimization of Gaussians, aiding in the reconstruction of unseen regions. In
summary, our method offers a globally consistent 3D scene within a
360^∘ perspective, providing an enhanced immersive experience over
existing techniques. Project website at: http://dreamscene360.github.io/
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要