A robust control-theory-based exploration strategy in deep reinforcement learning for virtual network embedding

Network slice management and, more generally, resource orchestration should be fully automated in 6G networks, as envisioned by the ETSI ENI. In this context, artificial intelligence (AI) and context-aware policies are certainly major options to move in this direction and to adapt service delivery to changing user needs, environmental conditions and business objectives. In this paper, we step towards this objective by addressing the problem of optimal placement of dynamic virtual networks through a self-adaptive learning-based strategy. These constantly evolving networks present, however, several challenges, mainly due to their stochastic nature, and the high dimensionality of the state and the action spaces. This curse of dimensionality requires, indeed, a broader exploration, which is not always compatible with a real-time execution in an operational network. Thus, we propose DQMC, a new strategy for virtual network embedding in mobile networks combining a Deep Reinforcement Learning (DRL) strategy, namely a Deep Q-Network (DQN), and Monte Carlo (MC). As learning is costly in time and computing resources, and sensitive to changes in the network, we suggest a control-theory-based techniques to dynamically leverage exploration in DQMC. This leads to fast, efficient, and sober learning compared to a Monte Carlo-based strategy. This also ensures a reliable solution even in the case of a change in the requests’ sizes or a node’s failure, showing promising perspectives for solutions combining control-theory and machine learning.