Delay-Optimized Multi-User VR Streaming via End-Edge Collaborative Neural Frame Interpolation

In this article, with the objective of significantly increasing the frame rate of virtual reality (VR) videos, we design an efficient end-edge collaborative VR streaming system which consists of three modules: frame similarity analysis, offloading decision making, and collaborative frame interpolation. In specific, frame similarity analysis tries to eliminate redundant frames based on perceived quality assessment, so that the required number of interpolated frames can be reduced without deteriorating visual quality. Then, an end-to-end (E2E) delay optimization problem is formulated to obtain the optimal offloading strategy, by balancing the transmission and computing burden of neural frame interpolation via end-edge collaboration. Furthermore, the E2E delay of the proposed system is theoretically analyzed based on queuing theory. Our analysis reveals that, the proposed collaborative distribution of interpolation tasks between edge and end devices are effective to achieve the minimal E2E delay of streaming VR videos. Extensive experimental results demonstrate that the proposed system can significantly improve the frame rate of VR videos, while maintaining timely VR content delivery in various networking conditions.