Hyperspectral Image Classification Based on Interactive Transformer and CNN With Multilevel Feature Fusion Network

Due to the powerful feature information mining ability of deep learning, models such as convolutional neural network (CNN) and Transformer have gained a certain progress in hyperspectral image classification (HSIC). Characteristically, the CNN is good at extracting local information, but it has the limitation of insufficient receptive field. While the Transformer has the advantage of global representation, it ignores local details to some extent. Therefore, this letter proposes an interactive Transformer and CNN with a multilevel feature fusion network (ITCNet) for HSIC. Specifically, in the image-based framework, features with different perceptual fields and depths are extracted interactively by a multilayer Transformer and CNN, then fused through a multilevel feature fusion module for class prediction. Experimental results on two real datasets verify its efficiency, with improvements over other related methods.