Automatic Estimation of Action Unit Intensities and Inference of Emotional Appraisals

The development of a two-stage approach for appraisal inference from automatically detected Action Unit (AU) intensities in recordings of human faces is described. AU intensity estimation is based on a hybrid approach fusing information from an individually fitted mesh model of the faces and texture information. Evaluation results for two datasets and a comparison against a state-of-the-art system, namely OpenFace are provided. In the second stage, the emotional appraisals novelty, valence and control are predicted from estimated AU intensities by linear regressions. Prediction performance is evaluated based on face recordings from a market research study, which were rated by human observers in terms of perceived appraisals. Predictions of valence and control from automatically estimated AU intensities closely match those obtained from manually coded AUs in terms of agreement with human observers, while novelty predictions lag somewhat behind. Overall, results highlight the flexibility and interpretability of a two-stage approach to emotion inference.