Pyramid hierarchical network for multispectral pan-sharpening

Pan-sharpening aims to fuse high spatial-resolution panchromatic images (PAN) and low spatial-resolution multispectral images (MS) into high spatial-resolution multispectral images (HRMS).We propose a pyramid hierarchical multi-spectral fusion network, called PH-Net which can automatically fuse MS images and PAN images to generate corresponding HRMS images. The architecture is based on the U-Net network. First, a multi-level receptive field is realised by constructing an input pyramid. Then, hierarchical features are extracted from the encoder, decoder, and input pyramid. Finally, the rich hierarchical features are used to calculate the residual error between the MS image and the corresponding HRMS image. The learned residual error is inserted into the MS image to obtain the final high spatial-resolution multispectral image. To demonstrate the effectiveness of each component in the network architecture, we conducted an ablation study. In addition, thanks to the design of the multi-layer architecture, model training does not require a large dataset, which greatly improves the training speed and significantly improves the generalisability and ease of deployment of this work in the field of remote sensing images. Through qualitative and quantitative experiments, we proved that the proposed method is superior to current advanced methods.