Improving reference picture resampling (RPR) for future video coding

Considering industry needs for further coding efficiency improvements, the Joined Exploration Team on Video (JVET) established by ITU-T and MPEG for standardizing VVC, has developed a new Enhanced Compression Model (ECM) based on VVC which is a common area for testing future video coding algorithms. The Versatile Video Coding (VVC) supports Reference Picture Resampling (RPR) to change frame resolution without inserting an Instantaneous Decoder Refresh (IDR) or Intra Random Access Picture (IRAP). This feature is particularly well adapted to video streaming and low delay scenarios since it allows seamless frame-based bit-rate adaptation, whereas traditional techniques based on stream switching between coded video chunks at fixed resolution can generate bitrate leaps. ECM implements several new tools that improve the coding efficiency compared to VVC, but some of them were not designed to support RPR. In this paper, we first discuss some necessary adaptations to implement RPR in ECM for these new coding tools. At low bit rate, RPR may improve the coding performance of ECM for luma component, and the coding complexity is reduced. However, RPR may show PSNR drop for chroma component because it performs an additional down-scaling filtering on samples that were already filtered from the original canonical 4:4:4 content to create the 4:2:0 format. Then, in a second part, some modifications of RPR to re-scale luma and chroma differently are proposed. It is shown that it improves ECM efficiency in the context of both super-resolution and low-delay coding use cases.