A Novel Approach for Automatic Program Repair using Round-Trip Translation with Large Language Models
CoRR(2024)
摘要
Research shows that grammatical mistakes in a sentence can be corrected by
translating it to another language and back using neural machine translation
with language models. We investigate whether this correction capability of
Large Language Models (LLMs) extends to Automatic Program Repair (APR). Current
generative models for APR are pre-trained on source code and fine-tuned for
repair. This paper proposes bypassing the fine-tuning step and using Round-Trip
Translation (RTT): translation of code from one programming language to another
programming or natural language, and back. We hypothesize that RTT with LLMs
restores the most commonly seen patterns in code during pre-training, i.e.,
performs a regression toward the mean, which removes bugs as they are a form of
noise w.r.t. the more frequent, natural, bug-free code in the training data. To
test this hypothesis, we employ eight recent LLMs pre-trained on code,
including the latest GPT versions, and four common program repair benchmarks in
Java. We find that RTT with English as an intermediate language repaired 101 of
164 bugs with GPT-4 on the HumanEval-Java dataset. Moreover, 46 of these are
unique bugs that are not repaired by other LLMs fine-tuned for APR. Our
findings highlight the viability of round-trip translation with LLMs as a
technique for automated program repair and its potential for research in
software engineering.
Keywords: automated program repair, large language model, machine translation
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要