Explainable Object Detection in the Field of Search and Rescue Robotics

State-of-the-art object detection in search and rescue robotics relies on CNNs, which reach excellent scores but lack explainability due to their “black box” characteristic. In such a domain, mission failure or misdetections can have drastic consequences. Therefore research should strive to increase the explainability regarding the CNNs’ classification strategies. In this paper, existing methods for object detection are applied and investigated in the context of search and rescue robotics in order to compose a fully explainable pipeline. Unlike existing object detection methods, the presented method is based on an exhaustive model investigation concerning post-hoc explainability. The method is applied to detecting handwheels of gate-valves, with a post-hoc analysis of the classification strategies learned by the object detector. In order to train and test the object detection model, a novel dataset is composed, including 2447 images, nine handwheel types, and 6696 annotations. Five CNN object detectors (R-CNN, Faster R-CNN, RetinaNet, YOLOv5s, and SSD) are compared based on mAP0.5 and mAP0.5:0.95. Two CNN object detectors’ classification strategies were investigated with the SpRAy method. Comparing mAP0.5, mAP0.5:0.95, and inference times reveals that YOLOv5s is the superior model across all categories. SpRAy analysis of R-CNN and YOLOv5s did not reveal any abnormal classification strategies, which indicates a well-balanced dataset. YOLOv5s appears to have learned different classification strategies for different handwheel types. Our handwheels dataset is available at: https://www.kaggle.com/hoenigpet/handwheels-for-gatevalves .