Human-multimodal deep learning collaboration in 'precise' diagnosis of lupus erythematosus subtypes and similar skin diseases

BackgroundLupus erythematosus (LE) is a spectrum of autoimmune diseases. Due to the complexity of cutaneous LE (CLE), clinical skin image-based artificial intelligence is still experiencing difficulties in distinguishing subtypes of LE.ObjectivesWe aim to develop a multimodal deep learning system (MMDLS) for human-AI collaboration in diagnosis of LE subtypes.MethodsThis is a multi-centre study based on 25 institutions across China to assist in diagnosis of LE subtypes, other eight similar skin diseases and healthy subjects. In total, 446 cases with 800 clinical skin images, 3786 multicolor-immunohistochemistry (multi-IHC) images and clinical data were collected, and EfficientNet-B3 and ResNet-18 were utilized in this study.ResultsIn the multi-classification task, the overall performance of MMDLS on 13 skin conditions is much higher than single or dual modals (Sen = 0.8288, Spe = 0.9852, Pre = 0.8518, AUC = 0.9844). Further, the MMDLS-based diagnostic-support help improves the accuracy of dermatologists from 66.88% +/- 6.94% to 81.25% +/- 4.23% (p = 0.0004).ConclusionsThese results highlight the benefit of human-MMDLS collaborated framework in telemedicine by assisting dermatologists and rheumatologists in the differential diagnosis of LE subtypes and similar skin diseases.