Conditional and Modal Reasoning in Large Language Models
CoRR(2024)
摘要
The reasoning abilities of large language models (LLMs) are the topic of a
growing body of research in artificial intelligence and cognitive science. In
this paper, we probe the extent to which a dozen LLMs are able to distinguish
logically correct inferences from logically fallacious ones. We focus on
inference patterns involving conditionals (e.g., 'If Ann has a queen, then Bob
has a jack') and epistemic modals (e.g., 'Ann might have an ace', 'Bob must
have a king'). These inference patterns have been of special interest to
logicians, philosophers, and linguists, since they plausibly play a central
role in human reasoning. Assessing LLMs on these inference patterns is thus
highly relevant to the question of how much the reasoning abilities of LLMs
match those of humans. Among the LLMs we tested, all but GPT-4 often make basic
mistakes with conditionals. Moreover, even GPT-4 displays logically
inconsistent judgments across inference patterns involving epistemic modals.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要