Prediction of Engagement from Facial Expressions: Effect of Dynamic Factors

Estimating students’ engagement while studying is important to increase learning outcomes. However, it is difficult for teachers to pay attention to all students throughout a class, especially in online classes. We developed a method to estimate the engagement of learners, evaluating task performance from their facial expressions, where we defined engagement level by task performance, that is, the time required to execute a task in an experiment. Participants’ task performance was measured as the time of calculation (addition or subtraction) in mind, and we recorded their face images while executing calculation. We trained a machine learning model to predict calculation times from facial features. The method succeeded in predicting calculation time with a certain degree of accuracy, suggesting that facial features are useful for estimating mental states while studying. Furthermore, we showed that changes in facial expression are more effective to the prediction of calculation time than average maximum and standard deviation of facial expressions.