Policy Improvement using Language Feedback Models
CoRR(2024)
摘要
We introduce Language Feedback Models (LFMs) that identify desirable
behaviour - actions that help achieve tasks specified in the instruction - for
imitation learning in instruction following. To train LFMs, we obtain feedback
from Large Language Models (LLMs) on visual trajectories verbalized to language
descriptions. First, by using LFMs to identify desirable behaviour to imitate,
we improve in task-completion rate over strong behavioural cloning baselines on
three distinct language grounding environments (Touchdown, ScienceWorld, and
ALFWorld). Second, LFMs outperform using LLMs as experts to directly predict
actions, when controlling for the number of LLM output tokens. Third, LFMs
generalize to unseen environments, improving task-completion rate by 3.5-12.0
through one round of adaptation. Finally, LFM can be modified to provide
human-interpretable feedback without performance loss, allowing human
verification of desirable behaviour for imitation learning.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要