IOFusion: instance segmentation and optical-flow guided 3D reconstruction in dynamic scenes
The Visual Computer(2024)
摘要
To improve the accuracy of camera pose estimation for RGBD-based 3D reconstruction in dynamic scenes, a method based on instance segmentation and optical flow is proposed. Firstly, instance segmentation is used to detect objects, and a semantic map is constructed by removing non-rigid objects. Secondly, motion residual is calculated by optical flow and camera flow to detect dynamic rigid objects, and nonlinear optimization is used to track the dynamic rigid objects extracted from the semantic map. Thirdly, after removing features of non-rigid objects and dynamic rigid ones in each frame, the remaining features are used to optimize the camera pose. Finally, a TSDF model is used to reconstruct the static background, and point clouds are used to reconstruct dynamic rigid objects. Experiments on TUM and Bonn datasets show that the method produces better camera poses than the current state-of-the-art methods in most dynamic scenes. Ablation experiments on Bonn dataset show that retaining features of static rigid objects significantly improves camera pose estimation precision. The annotated datasets and the source code are available at https://github.com/CodingMaplee/IOFusion/tree/main .
更多查看译文
关键词
3D reconstruction,Instance segmentation,Optical flow,Dynamic scene
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要