LiveNVS: Neural View Synthesis on Live RGB-D Streams.
ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia(2023)
摘要
Existing real-time RGB-D reconstruction approaches, like Kinect Fusion, lack
real-time photo-realistic visualization. This is due to noisy, oversmoothed or
incomplete geometry and blurry textures which are fused from imperfect depth
maps and camera poses. Recent neural rendering methods can overcome many of
such artifacts but are mostly optimized for offline usage, hindering the
integration into a live reconstruction pipeline.
In this paper, we present LiveNVS, a system that allows for neural novel view
synthesis on a live RGB-D input stream with very low latency and real-time
rendering. Based on the RGB-D input stream, novel views are rendered by
projecting neural features into the target view via a densely fused depth map
and aggregating the features in image-space to a target feature map. A
generalizable neural network then translates the target feature map into a
high-quality RGB image. LiveNVS achieves state-of-the-art neural rendering
quality of unknown scenes during capturing, allowing users to virtually explore
the scene and assess reconstruction quality in real-time.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要