OneChart: Purify the Chart Structural Extraction via One Auxiliary Token
arxiv(2024)
摘要
Chart parsing poses a significant challenge due to the diversity of styles,
values, texts, and so forth. Even advanced large vision-language models (LVLMs)
with billions of parameters struggle to handle such tasks satisfactorily. To
address this, we propose OneChart: a reliable agent specifically devised for
the structural extraction of chart information. Similar to popular LVLMs,
OneChart incorporates an autoregressive main body. Uniquely, to enhance the
reliability of the numerical parts of the output, we introduce an auxiliary
token placed at the beginning of the total tokens along with an additional
decoder. The numerically optimized (auxiliary) token allows subsequent tokens
for chart parsing to capture enhanced numerical features through causal
attention. Furthermore, with the aid of the auxiliary token, we have devised a
self-evaluation mechanism that enables the model to gauge the reliability of
its chart parsing results by providing confidence scores for the generated
content. Compared to current state-of-the-art (SOTA) chart parsing models,
e.g., DePlot, ChartVLM, ChartAst, OneChart significantly outperforms in Average
Precision (AP) for chart structural extraction across multiple public
benchmarks, despite enjoying only 0.2 billion parameters. Moreover, as a chart
parsing agent, it also brings 10
(LLaVA-1.6) in the downstream ChartQA benchmark.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要