IMO: Greedy Layer-Wise Sparse Representation Learning for Out-of-Distribution Text Classification with Pre-trained Models
arxiv(2024)
摘要
Machine learning models have made incredible progress, but they still
struggle when applied to examples from unseen domains. This study focuses on a
specific problem of domain generalization, where a model is trained on one
source domain and tested on multiple target domains that are unseen during
training. We propose IMO: Invariant features Masks for Out-of-Distribution text
classification, to achieve OOD generalization by learning invariant features.
During training, IMO would learn sparse mask layers to remove irrelevant
features for prediction, where the remaining features keep invariant.
Additionally, IMO has an attention module at the token level to focus on tokens
that are useful for prediction. Our comprehensive experiments show that IMO
substantially outperforms strong baselines in terms of various evaluation
metrics and settings.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要