A Novel Capsule Aggregation Framework for Natural Language Inference

Recent advances have advocated the use of complex attention mechanism to capture interactive information between premise and hypothesis in Natural Language Inference (NLI). However, few studies have focused on the further processing of matching feature, i.e. information aggregation. In this paper, we first investigate a novel capsule network for NLI, referred as Gcap. Gcap utilizes a gated enhanced fusion operation to obtain richer features between massive soft alignment information. Then the capsule aggregates those features through routing algorithms. Benefit from the routing mechanism of the capsule network, Gcap can dynamically generate feature vectors for subsequent classifier. Evaluation results demonstrate that our model achieves accuracy of 89.1%, 88.2% and 79.6% (79.3%) on SNLI, SciTail and MultiNLI datasets respectively, which outperforms the strong baseline with gains of 0.2%, 1.4% and 0.3% (0.6%). In particular, we compare the runtime inference efficiency of BERT and our model. Our model can attain up to 33.3\(\times \) speedup in online inference time. Thanks to dynamic aggregation, Gcap shows a strong ability to distinguish those cases that are easily confused.