Learning Latent Geometric Consistency for 6D Object Pose Estimation in Heavily Cluttered Scenes

•A dual-stream deep learning network is proposed for 6D object pose estimation.•Latent geometric consistency is learned to enforce structural constraints.•The proposed scheme is robust to heavy occlusion and segmentation errors.•Pairwise dense features are generated from two frames of different viewing angles.