MathGenie: Generating Synthetic Data with Question Back-translation for Enhancing Mathematical Reasoning of LLMs
CoRR(2024)
摘要
Large language models (LLMs) have exhibited great potential in mathematical
reasoning. However, there remains a performance gap in this area between
existing open-source models and closed-source models such as GPT-4. In this
paper, we introduce MathGenie, a novel method for generating diverse and
reliable math problems from a small-scale problem-solution dataset (denoted as
seed data). We augment the ground-truth solutions of our seed data and train a
back-translation model to translate the augmented solutions back into new
questions. Subsequently, we generate code-integrated solutions for the new
questions. To ensure the correctness of the code-integrated solutions, we
employ rationale-based strategy for solution verification. Various pretrained
models, ranging from 7B to 70B, are trained on the newly curated data to test
the effectiveness of the proposed augmentation technique, resulting in a family
of models known as MathGenieLM. These models consistently outperform previous
open-source models across five representative mathematical reasoning datasets,
achieving state-of-the-art performance. In particular, MathGenieLM-InternLM2
achieves an accuracy of 87.7
overall score among open-source language models.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要