Charting New Territories: Exploring the Geographic and Geospatial Capabilities of Multimodal LLMs
CoRR(2023)
摘要
Multimodal large language models (MLLMs) have shown remarkable capabilities
across a broad range of tasks but their knowledge and abilities in the
geographic and geospatial domains are yet to be explored, despite potential
wide-ranging benefits to navigation, environmental research, urban development,
and disaster response. We conduct a series of experiments exploring various
vision capabilities of MLLMs within these domains, particularly focusing on the
frontier model GPT-4V, and benchmark its performance against open-source
counterparts. Our methodology involves challenging these models with a
small-scale geographic benchmark consisting of a suite of visual tasks, testing
their abilities across a spectrum of complexity. The analysis uncovers not only
where such models excel, including instances where they outperform humans, but
also where they falter, providing a balanced view of their capabilities in the
geographic domain. To enable the comparison and evaluation of future models,
our benchmark will be publicly released.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要