Conceptual and Unbiased Reasoning in Language Models
arxiv(2024)
摘要
Conceptual reasoning, the ability to reason in abstract and high-level
perspectives, is key to generalization in human cognition. However, limited
study has been done on large language models' capability to perform conceptual
reasoning. In this work, we bridge this gap and propose a novel
conceptualization framework that forces models to perform conceptual reasoning
on abstract questions and generate solutions in a verifiable symbolic space.
Using this framework as an analytical tool, we show that existing large
language models fall short on conceptual reasoning, dropping 9
various benchmarks compared to direct inference methods. We then discuss how
models can improve since high-level abstract reasoning is key to unbiased and
generalizable decision-making. We propose two techniques to add trustworthy
induction signals by generating familiar questions with similar underlying
reasoning paths and asking models to perform self-refinement. Experiments show
that our proposed techniques improve models' conceptual reasoning performance
by 8
inductive biases.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要