CEFusion: an Infrared and Visible Image Fusion Network Based on Cross-Modal Multi-Granularity Information Interaction and Edge Guidance

Infrared and visible image fusion (IVF) aims to generate a fused image with abundant texture details and salient thermal radiation targets, which can not only preserve the necessary scene information for traffic vision tasks, but also highlight the imperceptible targets that are crucial in intelligent transportation system (ITS). However, the existing image fusion methods often lack the information interactions between cross-modal features and among cross-granularity features, and they usually ignore the importance of edge information to the image, which affects the quality of the fused image. To this end, this study proposes an IVF network based on cross-modal multi-granularity information interaction and edge guidance, termed as CEFusion. On the one hand, a triple-branch scene fidelity module is designed to fuse the different modal features extracted by the encoder. This module can adequately mine difference information and infrared salient information of the cross-modal features through cross-modal information interaction. On the other hand, a progressive cross-granularity interaction feature enhancement module is employed to achieve the information interaction among cross-granularity features, which can further enrich the texture and structure information in the fused features. In addition, a novel edge loss function is proposed to guide the network to retain the edge information from source images. Extensive comparative and generalization experiments demonstrate that our CEFusion superior to the state-of-the-art methods in preserving texture details and thermal radiation targets. More importantly, the performance of our method in the high-level vision task suggests that it can provide reliable assistance for the ITS applications.