Exploring Diverse In-Context Configurations for Image Captioning
arXiv (Cornell University)(2023)
摘要
After discovering that Language Models (LMs) can be good in-context few-shot
learners, numerous strategies have been proposed to optimize in-context
sequence configurations. Recently, researchers in Vision-Language (VL) domains
also develop their few-shot learners, while they only use the simplest way,
ie., randomly sampling, to configure in-context image-text pairs. In order to
explore the effects of varying configurations on VL in-context learning, we
devised four strategies for image selection and four for caption assignment to
configure in-context image-text pairs for image captioning. Here Image
Captioning is used as the case study since it can be seen as the
visually-conditioned LM. Our comprehensive experiments yield two
counter-intuitive but valuable insights, highlighting the distinct
characteristics of VL in-context learning due to multi-modal synergy, as
compared to the NLP case. Furthermore, in our exploration of optimal
combination strategies, we observed an average performance enhancement of 20.9
of CIDEr scores compared to the baseline. The code is given in
https://github.com/yongliang-wu/ExploreCfg.
更多查看译文
关键词
in-context
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要