Self-Supervised Pre-Training Transformer for Seismic Data Denoising

Seismic exploration is a crucial method for studying underground geological structures and oil/gas resources. However, the presence of various noise sources during seismic wave propagation hinders accurate interpretation and imaging. To address this challenge, effective denoising methods are essential. In recent years, deep learning, particularly Convolutional Neural Networks (CNNs), has shown promise in seismic data processing. Nevertheless, CNNs have limitations in capturing long-range dependencies and global coherence. As an alternative, we propose a Transformer-based model called Seismic Data Denoising Transformer (SDT) for seismic signal processing. By leveraging self-attention mechanisms, the SDT model overcomes the limitations of CNNs and effectively captures long-range features for seismic signal reconstruction. We also introduce a novel self-supervised pretraining strategy using a large-scale dataset to further enhance performance. Experimental results demonstrate the advantages of SDT in complex seismic noise attenuation and preserving weak signal amplitudes. The proposed method exhibits promising potential for real-world seismic data applications.