Multimodal Fusion on Low-quality Data: A Comprehensive Survey
arxiv(2024)
摘要
Multimodal fusion focuses on integrating information from multiple modalities
with the goal of more accurate prediction, which has achieved remarkable
progress in a wide range of scenarios, including autonomous driving and medical
diagnosis. However, the reliability of multimodal fusion remains largely
unexplored especially under low-quality data settings. This paper surveys the
common challenges and recent advances of multimodal fusion in the wild and
presents them in a comprehensive taxonomy. From a data-centric view, we
identify four main challenges that are faced by multimodal fusion on
low-quality data, namely (1) noisy multimodal data that are contaminated with
heterogeneous noises, (2) incomplete multimodal data that some modalities are
missing, (3) imbalanced multimodal data that the qualities or properties of
different modalities are significantly different and (4) quality-varying
multimodal data that the quality of each modality dynamically changes with
respect to different samples. This new taxonomy will enable researchers to
understand the state of the field and identify several potential directions. We
also provide discussion for the open problems in this field together with
interesting future research directions.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要