Almond defect and freshness inspection system using hyperspectral imaging and deep learning techniques

Almonds, recognized as a nutritious and low-calorie snack rich in unsaturated fats and essential nutrients, are primarily imported from California. Despite undergoing grading control by the United States Department of Agriculture (USDA), concerns persist regarding the presence of defective kernels. Manual hand-picking is the conventional method for almond processing; however, prolonged picking can reduce accuracy due to visual fatigue. Consumers prioritize freshness due to fungi’s potential for aflatoxin production, emphasizing the need for vigilant almond consumption. In response to these challenges, this study presents an innovative approach that employs a lightweight VGG16 and an enhanced MobileNet model for identifying defects based on RGB images of almonds. This method effectively classifies broken, scratched, and malformed kernels. Additionally, hyperspectral imaging (HSI) data in the 400–1000 nm range were collected to distinguish insect-damaged from undamaged kernels and assess almond freshness. For comparative analysis, various band selection algorithms were applied to extract distinct spectral bands. For insect damage identification, we propose a merged CNN model created by simultaneously combining 1D- and 2D-CNNs, allowing for the concurrent extraction of spectral and spatial characteristics. This approach achieved a remarkable ACC of 0.9905 and a Kappa value of 0.9521. Meanwhile, in evaluating freshness, PCA was employed to select the first 30 principal components as vital bands. Our proposed lightweight 1D-CNN delivered an ACC of 0.9768 and a Kappa value of 0.9527 in freshness assessment.