ScribbleGen: Generative Data Augmentation Improves Scribble-supervised Semantic Segmentation
arxiv(2023)
摘要
Recent advances in generative models, such as diffusion models, have made
generating high-quality synthetic images widely accessible. Prior works have
shown that training on synthetic images improves many perception tasks, such as
image classification, object detection, and semantic segmentation. We are the
first to explore generative data augmentations for scribble-supervised semantic
segmentation. We propose ScribbleGen, a generative data augmentation method
that leverages a ControlNet diffusion model conditioned on semantic scribbles
to produce high-quality training data. However, naive implementations of
generative data augmentations may inadvertently harm the performance of the
downstream segmentor rather than improve it. We leverage classifier-free
diffusion guidance to enforce class consistency and introduce encode ratios to
trade off data diversity for data realism. Using the guidance scale and encode
ratio, we can generate a spectrum of high-quality training images. We propose
multiple augmentation schemes and find that these schemes significantly impact
model performance, especially in the low-data regime. Our framework further
reduces the gap between the performance of scribble-supervised segmentation and
that of fully-supervised segmentation. We also show that our framework
significantly improves segmentation performance on small datasets, even
surpassing fully-supervised segmentation. The code is available at
https://github.com/mengtang-lab/scribblegen.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要