LLMRefine: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedback
arxiv(2023)
摘要
Recent large language models (LLM) are leveraging human feedback to improve
their generation quality. However, human feedback is costly to obtain,
especially during inference. In this work, we propose LLMRefine, an inference
time optimization method to refine LLM's output. The core idea is to use a
learned fine-grained feedback model to pinpoint defects and guide LLM to refine
them iteratively. Using original LLM as a proposal of edits, LLMRefine searches
for defect-less text via simulated annealing, trading off the exploration and
exploitation. We conduct experiments on three text generation tasks, including
machine translation, long-form question answering (QA), and topical
summarization. LLMRefine consistently outperforms all baseline approaches,
achieving improvements up to 1.7 MetricX points on translation tasks, 8.1
ROUGE-L on ASQA, 2.2 ROUGE-L on topical summarization.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要