Enabling Intelligent Interactions between an Agent and an LLM: A Reinforcement Learning Approach
CoRR(2023)
摘要
Large language models (LLMs) encode a vast amount of world knowledge acquired
from massive text datasets. Recent studies have demonstrated that LLMs can
assist an embodied agent in solving complex sequential decision making tasks by
providing high-level instructions. However, interactions with LLMs can be
time-consuming. In many practical scenarios, they require a significant amount
of storage space that can only be deployed on remote cloud server nodes.
Additionally, using commercial LLMs can be costly since they may charge based
on usage frequency. In this paper, we explore how to enable intelligent
cost-effective interactions between the agent and an LLM. We find that this
problem can be naturally formulated by a Markov decision process (MDP), and
propose When2Ask, a reinforcement learning based approach that learns when it
is necessary to query LLMs for high-level instructions to accomplish a target
task. Experiments on MiniGrid and Habitat environments that entail planning
sub-goals demonstrate that When2Ask learns to solve target tasks with only a
few necessary interactions with an LLM, and significantly reduces interaction
costs in testing environments compared with baseline methods. Experiment
results also suggest that by learning a mediator model to interact with the
LLM, the agent's performance becomes more robust against partial observability
of the environment. Our code is available at
https://github.com/ZJLAB-AMMI/LLM4RL.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要