Unified-modal Salient Object Detection via Adaptive Prompt Learning
arxiv(2023)
摘要
Existing single-modal and multi-modal salient object detection (SOD) methods
focus on designing specific architectures tailored for their respective tasks.
However, developing completely different models for different tasks leads to
labor and time consumption, as well as high computational and practical
deployment costs. In this paper, we attempt to address both single-modal and
multi-modal SOD in a unified framework called UniSOD, which fully exploits the
overlapping prior knowledge between different tasks. Nevertheless, assigning
appropriate strategies to modality variable inputs is challenging. To this end,
UniSOD learns modality-aware prompts with task-specific hints through adaptive
prompt learning, which are plugged into the proposed pre-trained baseline SOD
model to handle corresponding tasks, while only requiring few learnable
parameters compared to training the entire model. Each modality-aware prompt is
generated from a switchable prompt generation block, which adaptively performs
structural switching based on single-modal and multi-modal inputs without human
intervention. Through end-to-end joint training, UniSOD achieves overall
performance improvement on 14 benchmark datasets for RGB, RGB-D, and RGB-T SOD,
which demonstrates that our method effectively and efficiently unifies
single-modal and multi-modal SOD tasks.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要