DELTA: Decoupling Long-Tailed Online Continual Learning
arxiv(2024)
摘要
A significant challenge in achieving ubiquitous Artificial Intelligence is
the limited ability of models to rapidly learn new information in real-world
scenarios where data follows long-tailed distributions, all while avoiding
forgetting previously acquired knowledge. In this work, we study the
under-explored problem of Long-Tailed Online Continual Learning (LTOCL), which
aims to learn new tasks from sequentially arriving class-imbalanced data
streams. Each data is observed only once for training without knowing the task
data distribution. We present DELTA, a decoupled learning approach designed to
enhance learning representations and address the substantial imbalance in
LTOCL. We enhance the learning process by adapting supervised contrastive
learning to attract similar samples and repel dissimilar (out-of-class)
samples. Further, by balancing gradients during training using an equalization
loss, DELTA significantly enhances learning outcomes and successfully mitigates
catastrophic forgetting. Through extensive evaluation, we demonstrate that
DELTA improves the capacity for incremental learning, surpassing existing OCL
methods. Our results suggest considerable promise for applying OCL in
real-world applications.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要