BLINK: Multimodal Large Language Models Can See but Not Perceive
arxiv(2024)
摘要
We introduce Blink, a new benchmark for multimodal language models (LLMs)
that focuses on core visual perception abilities not found in other
evaluations. Most of the Blink tasks can be solved by humans "within a blink"
(e.g., relative depth estimation, visual correspondence, forensics detection,
and multi-view reasoning). However, we find these perception-demanding tasks
cast significant challenges for current multimodal LLMs because they resist
mediation through natural language. Blink reformats 14 classic computer vision
tasks into 3,807 multiple-choice questions, paired with single or multiple
images and visual prompting. While humans get 95.70
is surprisingly challenging for existing multimodal LLMs: even the
best-performing GPT-4V and Gemini achieve accuracies of 51.26
13.17
abilities have not "emerged" yet in recent multimodal LLMs. Our analysis also
highlights that specialist CV models could solve these problems much better,
suggesting potential pathways for future improvements. We believe Blink will
stimulate the community to help multimodal LLMs catch up with human-level
visual perception.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要