Remote Sensing Scene Classification with Multi-task Learning

Remote sensing scene classification is a fundamental task for understanding the high-resolution remote sensing images and has achieved great progress thanks to the deep neural networks. However, the deep neural networks heavily depend on the large-scale training samples, while sufficient manually annotated data is often unavailable. In this paper, a Multi-Task Scene Classification Network (MTSCN) is proposed to address the label consumption dilemma by using annotated information from other related tasks. The MTSCN simultaneously conducts multiple tasks so that scene classification task can leverage information shared by other tasks. The MTSCN consists of a shared branch and multiple task-specific branches with each for one task. First, the shared branch extracts shared features for all tasks. Then, the task-specific branch adapts shared features into task-specific features to finish the classification task. Such architecture allows complementary information shared among tasks as well as avoids task interference by selecting specific information for a different task. The method is evaluated on two multi-task learning scenarios with respect to accuracy and the number of network parameters. The results show that the proposed method is efficient not only in terms of data but also in terms of network parameters.