DeepVNP:Virtual Network Placing with Deep Reinforcement Learning in Industrial IoT

The diversity of devices, systems, and applications imposes stringent requirements on the Industrial Internet of Things (IIoT) regarding agility, reliability, and delay sensitivity. Network function virtualization (NFV) can provide on-demand service and flexible resource management for the IIoT. Of course, a highefficiency and time-saving NFV placement solution is critical for IIoT. Information processing requests in an actual industrial scenario is generally continuous and dynamic. In addition, industrial information systems pay more attention to the real-time nature of the information. Considering the above challenges and the fact that most previous optimization-based solutions cannot cope with the characteristics of dynamic requests, we propose a deep reinforcement learning-based method to solve the virtual network function (VNF) placement problem, called DeepVNP, which automatically places the VNF according to the current physical network state, aiming to reduce the placement cost and improve the profit of the NFV system. In addition, the Information Age (AoI) is introduced to measure the real-time freshness of information, and it is naturally integrated with delay constraints. Through simulations, we evaluate the convergence and performance of DeepVNP. Numerical results show that, compared with several existing solutions, DeepVNP performs well in terms of request acceptance rate, resource utilization, total system cost, and average AoI.