Multi-Model Deep Learning for Cloud Resources Prediction to Support Proactive Workflow Adaptation

Scientific workflows are complex, resource intensive, dynamic in nature and require elastic cloud resources. To support these requirements, cloud resources' prediction schemes forecast resource scarcity and therefore support proactive workflow adaptation. In this paper, we propose a proactive workflow adaptation approach supported by a Deep Learning based prediction of cloud resources' usage. The model uses an algorithm to evaluate and privilege the most appropriate prediction model for resource utilization violations for each task of the workflow. Then, it recommends the proper adaptation actions to maintain the Quality of Service (QoS) for the entire workflow. Runtime monitoring of cloud resources data is continuously fed into Machine Learning models including GRU, LSTM, and Bi-directional LSTM for predicting the future task resource utilization values. The algorithm evaluates the resources' prediction using a number of metrics, such as RMSE, MAE, and MAPE. The prediction model achieving the highest accuracy is selected to determine the needed cloud resources. We conducted a series of experiments to evaluate our approach and the results demonstrate that the proposed Multi-Model predicts properly the cloud resource usage as well as suggesting their adaptation actions to guarantee the required workflow QoS.