RIS-Assisted UAV for Fresh Data Collection in 3D Urban Environments: A Deep Reinforcement Learning Approach

Dispatching flexible unmanned aerial vehicles (UAVs) to collect data from distributed Internet-of-Things devices (IoTDs) is expected to be a promising technology to support time-critical applications. However, in urban environments, the communication links between the UAV and IoTDs are prone to be frequently blocked by buildings, which severely impairs the freshness of information collected by the UAV. Thus, how to overcome urban building blockages and ensure fresh data collection is quite important but neglected in existing works. In this paper, for keeping the information fresh, we propose to utilize the reconfigurable intelligent surface (RIS) to assist the UAV in mitigating signal propagation impairments caused by building blockages. The formulated optimization problem is minimizing the age of information (AoI) of all IoTDs by jointly optimizing the UAV trajectory, IoTD scheduling and discrete phase shifts of the RIS. It is a mixed-integer non-convex problem as well as lacking the complete channel state information (CSI), thus using conventional optimization methods is intractable. To address this issue, we present an effective and robust deep reinforcement learning (DRL)-based scheme called “SAC-AO-RIS,” where a soft actor-critic (SAC) algorithm with recent-prioritized experience replay is designed for learning highly stable policies of UAV trajectory and IoTD scheduling, and an alternating optimization (AO) algorithm is leveraged for solving RIS phase shifts. Finally, simulation results demonstrate the effectiveness and superiority of our proposed scheme compared with other baseline approaches.