Deep learning based detection of osteophytes in radiographs and magnetic resonance imagings of the knee using 2D and 3D morphology

In this study, we investigated the discriminative capacity of knee morphology in automatic detection of osteophytes defined by the Osteoarthritis Research Society International atlas, using X-ray and magnetic resonance imaging (MRI) data. For the X-ray analysis, we developed a deep learning (DL) based model to segment femur and tibia. In case of MRIs, we utilized previously validated segmentations of femur, tibia, corresponding cartilage tissues, and menisci. Osteophyte detection was performed using DL models in four compartments: medial femur (FM), lateral femur (FL), medial tibia (TM), and lateral tibia (TL). To analyze the confounding effects of soft tissues, we investigated their morphology in combination with bones, including bones+cartilage, bones+menisci, and all the tissues. From X-ray-based 2D morphology, the models yielded balanced accuracy of 0.73, 0.69, 0.74, and 0.74 for FM, FL, TM, TL, respectively. Using 3D bone morphology from MRI, balanced accuracy was 0.80, 0.77, 0.71, and 0.76, respectively. The performance was higher than in 2D for all the compartments except for TM, with significant improvements observed for femoral compartments. Adding menisci or cartilage morphology consistently improved balanced accuracy in TM, with the greatest improvement seen for small osteophyte. Otherwise, the models performed similarly to bones-only. Our experiments demonstrated that MRI-based models show higher detection capability than X-ray based models for identifying knee osteophytes. This study highlighted the feasibility of automated osteophyte detection from X-ray and MRI data and suggested further need for development of osteophyte assessment criteria in addition to OARSI, particularly, for early osteophytic changes.