TAIL: Task-specific Adapters for Imitation Learning with Large Pretrained Models
arxiv(2023)
摘要
The full potential of large pretrained models remains largely untapped in
control domains like robotics. This is mainly because of the scarcity of data
and the computational challenges associated with training or fine-tuning these
large models for such applications. Prior work mainly emphasizes either
effective pretraining of large models for decision-making or single-task
adaptation. But real-world problems will require data-efficient, continual
adaptation for new control tasks. Recognizing these constraints, we introduce
TAIL (Task-specific Adapters for Imitation Learning), a framework for efficient
adaptation to new control tasks. Inspired by recent advancements in
parameter-efficient fine-tuning in language domains, we explore efficient
fine-tuning techniques – e.g., Bottleneck Adapters, P-Tuning, and Low-Rank
Adaptation (LoRA) – in TAIL to adapt large pretrained models for new tasks
with limited demonstration data. Our extensive experiments in large-scale
language-conditioned manipulation tasks comparing prevalent parameter-efficient
fine-tuning techniques and adaptation baselines suggest that TAIL with LoRA can
achieve the best post-adaptation performance with only 1% of the trainable
parameters of full fine-tuning, while avoiding catastrophic forgetting and
preserving adaptation plasticity in continual learning settings.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要