Continual Learning of Diffusion Models with Generative Distillation.
CoRR(2023)
摘要
Diffusion models are powerful generative models that achieve state-of-the-art
performance in tasks such as image synthesis. However, training them demands
substantial amounts of data and computational resources. Continual learning
would allow for incrementally learning new tasks and accumulating knowledge,
thus reusing already trained models would be possible. One potentially suitable
approach is generative replay, where a copy of a generative model trained on
previous tasks produces synthetic data that are interleaved with data from the
current task. However, standard generative replay applied to diffusion models
results in a catastrophic loss in denoising capabilities. In this paper, we
propose generative distillation, an approach that distils the entire reverse
process of a diffusion model. We demonstrate that our approach significantly
improves the continual learning performance of generative replay with only a
moderate increase in the computational costs.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要