Integrated 2D and 3D Images for Face Recognition

Abstract: This paper presents a feature-based face recognition system based on both 3D range data as well as 2D gray-level facial images. Ten 2D feature points and four 3D feature points are designed to be robust against changes of facial expressions and viewpoints and are described by Gabor filter responses in the 2D domain and Point Signature in the 3D domain. Localizing feature points in a new facial image is based on 3D-2D correspondence, average layout and corresponding Bunch (covering a wide range of possible variations on each point). Extracted shape features from 3D feature points and texture features from 2D feature points are first projected into their own subspace using PCA. In subspace, the corresponding shape and texture weight vectors are then integrated to form an augmented vector which is used to represent each facial image. For a given test facial image, the best match in the model library is identified according to a classifier. Similarity function and Support Vector Machine (SVM) are two types of classifiers considered in this work. Experimental results involving 20 persons with different facial expressions and extracted from different viewpoints have demonstrated the efficiency of our algorithm.