Sim-to-Real Grasp Detection with Global-to-Local RGB-D Adaptation
CoRR(2024)
摘要
This paper focuses on the sim-to-real issue of RGB-D grasp detection and
formulates it as a domain adaptation problem. In this case, we present a
global-to-local method to address hybrid domain gaps in RGB and depth data and
insufficient multi-modal feature alignment. First, a self-supervised rotation
pre-training strategy is adopted to deliver robust initialization for RGB and
depth networks. We then propose a global-to-local alignment pipeline with
individual global domain classifiers for scene features of RGB and depth images
as well as a local one specifically working for grasp features in the two
modalities. In particular, we propose a grasp prototype adaptation module,
which aims to facilitate fine-grained local feature alignment by dynamically
updating and matching the grasp prototypes from the simulation and real-world
scenarios throughout the training process. Due to such designs, the proposed
method substantially reduces the domain shift and thus leads to consistent
performance improvements. Extensive experiments are conducted on the
GraspNet-Planar benchmark and physical environment, and superior results are
achieved which demonstrate the effectiveness of our method.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要