When Moving Target Defense Meets Attack Prediction in Digital Twins: A Convolutional and Hierarchical Reinforcement Learning Approach

With rapid development of emerging technologies for Internet of Things (IoT), digital twins (DT) have been proposed to support a wide variety of applications. A mobile network is expected to be integrated with DT to form a DT mobile network (DTMN). Unfortunately, DTMN still faces security threats, which have attracted great research attention. Current defense mechanisms are mostly static, i.e., responding after attacks happening. To solve the aforementioned problem, moving target defense (MTD) has been proposed as an innovative solution. However, there exist three major challenges when applying MTD into DTMN. Firstly, less emphasis was paid to collaborative scheduling between multiple MTD schemes, which can improve the security of DTMN. Secondly, MTD schemes require lots of network resources, but few works focus on the time allocation of multiple MTD schemes to reduce network resource consumption. Thirdly, existing defense strategies only rely on current information, but do not consider future information. In this paper, we propose a collaborative mutation-based MTD (CM-MTD) in DTMN. We mainly consider two MTD schemes called host address mutation (HAM) and route mutation (RM), respectively, which adjust network properties and invalidate different stages of cyber kill chain. We firstly formulate a semi-Markov decision process (SMDP) to model time-varying security events and dynamic deployment of multiple MTD schemes. Then, security events are predicted by long short-term memory (LSTM), which are regarded as network states in SMDP. Next, infeasible actions that do not satisfy network constraints will be removed from the action space of the SMDP. Lastly, we design a hierarchical deep reinforcement learning algorithm for collaborative scheduling. Simulation results highlight the effectiveness of CM-MTD compared with baseline solutions.