3D Facial Expression Recognition Using Spiral Convolutions and Transformers

Facial expressions are a crucial aspect of nonverbal human communication. Recent advancements in 3D facial expression recognition have allowed for accurate analysis and interpretation of these expressions using three-dimensional facial data. In this work, we present a novel approach for dynamic facial expression recognition using sequences of 3D meshes. Unlike existing state-of-the-art methods that either rely on hand-designed feature descriptors or project the faces to 2D domain, our proposed method directly extracts spatio-temporal information from the 3D meshes in a fully learned manner. This is achieved by feeding input meshes from the sequence individually to a spatial auto-encoder, that uses spiral convolutions to extract spatial embedding. Then, the sequence of embeddings is processed by a temporal transformer to capture temporal context and perform expression classification. We evaluate the proposed method on two facial expression benchmarks: MUG database and BU-4DFE database. Our preliminary experimental results demonstrate the effectiveness of the proposed approach in facial expression recognition. However, we acknowledge that the quality of pre-processing and mesh registration significantly impacts the obtained results. Further investigation and refinement in these areas are crucial for achieving a more accurate and robust performance.