A Deep Learning Approach for Automatic Detection, Segmentation and Classification of Breast Lesions from Thermal Images

Purpose: This study investigates implementation of deep learning (DL) approaches to breast tumor recognition based on thermal images. We propose to utilize Mask R-CNN technique on images by first assigning bounding boxes and then creating a border for each tumor volume to differentiate it from adjacent tissues and structures. In this manner, thermal images can be handled by a single DL model to successfully perform detection, classification, and segmentation of normal and abnormal breast tissues. Methods: This study employs Mask R-CNN technique along with transfer learning models to accurately delineate breast volumes from adjacent tissues and structures. It is a novel study that uses a single DL model to carry out three steps of breast tumor diagnosis, namely detection, segmentation and classification of normal and abnormal tissues based on thermal images. Results: Two network architectures (ResNet-50 and ResNet-101) were trained for 60 epochs and were then evaluated based on their classification and segmentation performances. The testing process resulted in higher classification success for ResNet-50 backbone pretrained on COCO images (%97.1 accuracy). Detection and segmentation performances of this model were also higher with mAP of 0.921 and overlap score of 0.868. Conclusions: Our results indicate that the classification and segmentation performances of Mask R-CNN method on ResNet-50 architecture are better than the data reported in the literature for thermal breast image studies. It should be emphasized that our approach employs a single DL model that successfully performs detection, classification, and segmentation procedures for diagnosing normal and abnormal breast tissues.