Feature extraction and matching in content-based retrieval of functional magnetic resonance images

Functional Magnetic Resonance Imaging (fMRI) has become a widely used technique in neuroscience research. Brain regions corresponding to certain cognitive functionalities can be located by studying the intensity change in a series of 3D brain scans. Although fMRI has been widely studied, little attention has been paid to content-based (“content” means the explicit or implicit cognitive process) retrieval of images despite the existence of databases equipped with textual description (fMRIDC). Content-based retrieval is potentially useful in discovering brain activation patterns, and in diagnoses by comparing observed patterns with those of known diseases, leading to clinical applications. We conducted a comprehensive investigation of feature extraction and similarity measures used in several research communities (including information retrieval (IR), signal processing, and computer vision(CV)), to set up a content-based retrieval framework for a large, heterogeneous database. We developed methods for both hypothesis-based (stimulus known) and hypothesis-free (stimulus unknown) schemes. For the former, we adapted and extended an adaptive Finite Impulse Response (FIR) Model to get a more robust estimation of the activation level of brain regions. We then relaxed the assumption that the brain responds as a linear time-invariant (LTI) system, by using a 4-parameter ordinary differential equation to model brain responses. We then evaluated a number of similarity measures used in IR and CV, such as Latent Semantic Indexing (LSI), TFIDF, and Mahalanobis distance, etc. For the latter, we used a heuristic to select independent components with low mean temporal frequency, and applied a maximum weight bipartite matching technique to integrate component-level similarity and give a more robust retrieval performance. For feature selection, we found that an FIR model with a smoothing factor can improve retrieval performance significantly. For feature matching, a method similar to “dilation operators” used in image processing gives better and more robust retrieval performance than other methods.