Explaining the predictions of kernel SVM models for neuroimaging data analysis

Machine learning methods have shown great performance in many areas, including neuroimaging data analysis. However, model performance is only one objective in neuroimaging analysis. Gaining insight from the data is also critical in this field, such as identifying regions where detected signals are relevant to cognitive and diagnostic tasks. To fulfil this need, enabling the explainability of a model’s decision-making process is critical. Predictions of complex machine learning models are notoriously difficult to explain. This limits the use of complex models like kernel support vector machines (SVM) in neuroimaging analysis. Recently, several permutation-based methods have been developed to explain these complex models. However, the explanation results are affected by class-irrelevant features like suppressor variables and high background noise variables. This problem may also happen when explaining linear models. One possible reason is that the permutation process will produce unrealistic data instances when features are not independent, e.g. correlated. These unrealistic data instances will influence the explanation results. In neuroimaging analysis, the activation pattern, the estimated weight of the assumed generative model corresponding to the current classifier, is used to deal with this problem for linear models. This method does not rely on a permutation process but rather on the available data information. In this paper, we propose a novel method of Explanation through Activation Pattern (EAP) to explain the SVM models with different types of kernels for neuroimaging data analysis. Our method can generate a global feature importance score by estimating the activation pattern of kernel SVM models. We evaluate our method against three popular methods on both simulation datasets and a publicly available EEG/MEG dataset on visual tasks. The experimental results demonstrate that the proposed EAP method can provide explanations with low computational cost and is less affected by class-irrelevant features than the other three methods. In the experiment using the MEG/EEG dataset of visual tasks, the proposed EAP method provides agreement results with the brain’s electrical activity patterns reported in the literature on the visual tasks EEG/MEG data and is significantly faster than the other explanation methods.