Latent Dataset Distillation with Diffusion Models
CoRR(2024)
摘要
The efficacy of machine learning has traditionally relied on the availability
of increasingly larger datasets. However, large datasets pose storage
challenges and contain non-influential samples, which could be ignored during
training without impacting the final accuracy of the model. In response to
these limitations, the concept of distilling the information on a dataset into
a condensed set of (synthetic) samples, namely a distilled dataset, emerged.
One crucial aspect is the selected architecture (usually ConvNet) for linking
the original and synthetic datasets. However, the final accuracy is lower if
the employed model architecture differs from the model used during
distillation. Another challenge is the generation of high-resolution images,
e.g., 128x128 and higher. In this paper, we propose Latent Dataset Distillation
with Diffusion Models (LD3M) that combine diffusion in latent space with
dataset distillation to tackle both challenges. LD3M incorporates a novel
diffusion process tailored for dataset distillation, which improves the
gradient norms for learning synthetic images. By adjusting the number of
diffusion steps, LD3M also offers a straightforward way of controlling the
trade-off between speed and accuracy. We evaluate our approach in several
ImageNet subsets and for high-resolution images (128x128 and 256x256). As a
result, LD3M consistently outperforms state-of-the-art distillation techniques
by up to 4.8 p.p. and 4.2 p.p. for 1 and 10 images per class, respectively.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要