Adversarial Stacking Ensemble for Facial Landmark Tracking

Current approaches for facial landmark tracking predict facial landmarks through either a single tracker or an ensemble of trackers. However, the conventional ensemble is not designed for facial landmark tracking and can not capture spatial and temporal patterns of facial landmarks efficiently. In this paper, we propose to extend the conventional stacking with an adversarial training strategy to better suit the facial landmark tracking task. Specifically, the meta learner attempts to distinguish the predictions from the base learners with the ground truths, while the base learners attempt to confuse the meta learner by predicting landmarks close to the ground truths. The adversary between the two levels of learners forces them to fully capture the inherent spatial and temporal patterns of facial landmarks. Moreover, to promote the diversity of different base trackers, we design two classification tasks at both the feature level and prediction level. Experimental results on the 300VW dataset and the TF dataset demonstrate the effectiveness of our method, and we achieve state-of-the-art performances on both datasets.