Feature Splatting: Language-Driven Physics-Based Scene Synthesis and Editing
arxiv(2024)
摘要
Scene representations using 3D Gaussian primitives have produced excellent
results in modeling the appearance of static and dynamic 3D scenes. Many
graphics applications, however, demand the ability to manipulate both the
appearance and the physical properties of objects. We introduce Feature
Splatting, an approach that unifies physics-based dynamic scene synthesis with
rich semantics from vision language foundation models that are grounded by
natural language. Our first contribution is a way to distill high-quality,
object-centric vision-language features into 3D Gaussians, that enables
semi-automatic scene decomposition using text queries. Our second contribution
is a way to synthesize physics-based dynamics from an otherwise static scene
using a particle-based simulator, in which material properties are assigned
automatically via text queries. We ablate key techniques used in this pipeline,
to illustrate the challenge and opportunities in using feature-carrying 3D
Gaussians as a unified format for appearance, geometry, material properties and
semantics grounded on natural language. Project website:
https://feature-splatting.github.io/
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要