AGG: attention-based gated convolutional GAN with prior guidance for image inpainting

Image inpainting has made great achievements recently, but it is often tough to generate a semantically consistent image when faced with large missing areas in complex scenes. To address semantic and structural alignment in existing methods for image inpainting, this paper proposes an end-to-end attention-based gated convolution GAN with prior guidance named AGG, which designs the spatial and channel attention mechanisms for full extraction of semantic and structural features. Moreover, AGG constructs the attention-based upsampling module based on the channel attention to refine the feature map and capture more details from features of up-level low sizes. AGG uses the image contour as prior, allowing the gated convolution and attention mechanism may fill the image efficiently by focusing on the contour information. The attention-based gated convolution can effectively capture the global features and compensate for the limitations of the restricted receptive field of the naive convolution. Compared to other models, AGG generates images with finer outline features and no common problems such as the watermark and blur, which shows the best overall performance on the Paris StreetView, CelebA-HQ, and Places2 datasets. The best FID, LPIPS, PSNR, and SSIM values achieved by AGG are 2.18, 0.046, 30.82, and 0.951 on the CelebA-HQ dataset, with at least 3.21 https://github.com/Shawn-Yu-1/AGGNet .