DCR: Divide-and-Conquer Reasoning for Multi-choice Question Answering with LLMs
arxiv(2024)
摘要
Large language models (LLMs) have shown impressive performance in reasoning
benchmarks with the emergence of Chain-of-Thought (CoT), particularly in
multi-choice question (MCQ). However, current works equally resolve questions
regardless of the problem-solving difficulty, leading to an excessive focus on
simple items while insufficient attention on intricate ones. To address this
challenge, we propose a simple yet effective strategy, Divide and Conquer
Reasoning (DCR), to enhance the reasoning capability of LLMs for MCQs, as
inspired by human beings using heuristics to first categorize tasks and then
handle them separately. In particular, we first categorize questions into two
subsets based on confidence score (𝒞𝒮), which is estimated by
statistical frequency of generated answers. Subsequently, we propose Filter
Choices based Reasoning (FCR) to improve model performance on MCQs with low
(𝒞𝒮). Our experiments demonstrate that the proposed strategy only
costs 85
across nine datasets including arithmetic, commonsense, and logic reasoning
tasks. The code is at
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要