Unified CNN-LSTM for keyhole status prediction in PAW based on spatial-temporal features

Despite the high efficiency of keyhole plasma arc welding (K-PAW), it still has several deficiencies, such as narrow welding parameter ranges, easily disturbed welding process and instability of welding quality, etc. It is a significant prerequisite to predict the keyhole/penetration status accurately for maintaining the welding process stability and improving the welding quality. Most researchers have focused on visual inspection techniques and convolutional neural networks (CNN), establishing a mathematical model of the correlation between weld pool images and penetration status. While CNN could extract the features of single weld pool images, it is difficult to predict the evolution trend of the incoming moments. In this paper, a novel model based on CNN and LSTM (long-short term memory) was developed to extract both the spatial and temporal features of topside weld pool images, and consequently the complex keyhole behaviors were predicted and described. The comparative study is carried out on different models, i.e. the single CNN model, the single LSTM model, and the CNN-LSTM model that integrates both spatial features of single images and temporal features of sequence. The keyhole initialization and establishing period, as the typical and critical welding scenario in full-penetration welding, is investigate and compared, resulting in a predicted value that is close to reality. Furthermore, ahead prediction of keyhole status was adopted, maintaining over 80% accuracy even when predicting keyhole behaviors 2 s into the future. Consequently, the unified CNN-LSTM model effectively improves the prediction accuracy of keyhole/ penetration status, promising for intelligent K-PAW technology.