Self-Consistency Boosts Calibration for Math Reasoning
arxiv(2024)
摘要
Calibration, which establishes the correlation between accuracy and model
confidence, is important for LLM development. We design three off-the-shelf
calibration methods based on self-consistency (Wang et al., 2022) for math
reasoning tasks. Evaluation on two popular benchmarks (GSM8K and MathQA) using
strong open-source LLMs (Mistral and LLaMA2), our methods better bridge model
confidence and accuracy than existing methods based on p(True) (Kadavath et
al., 2022) or logit (Kadavath et al., 2022).
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要