Bayesian Optimization with Adaptive Kernels for Robot Control
ICRA(2024)
摘要
Active policy search combines the trial-and-error methodology from policy
search with Bayesian optimization to actively find the optimal policy. First,
policy search is a type of reinforcement learning which has become very popular
for robot control, for its ability to deal with complex continuous state and
action spaces. Second, Bayesian optimization is a sample efficient global
optimization method that uses a surrogate model, like a Gaussian process, and
optimal decision making to carefully select each sample during the optimization
process. Sample efficiency is of paramount importance when each trial involves
the real robot, expensive Monte Carlo runs, or a complex simulator. Black-box
Bayesian optimization generally assumes a cost function from a stationary
process, because nonstationary modeling is usually based on prior knowledge.
However, many control problems are inherently nonstationary due to their
failure conditions, terminal states and other abrupt effects. In this paper, we
present a kernel function specially designed for Bayesian optimization, that
allows nonstationary modeling without prior knowledge, using an adaptive local
region. The new kernel results in an improved local search (exploitation),
without penalizing the global search (exploration), as shown experimentally in
well-known optimization benchmarks and robot control scenarios. We finally show
its potential for the design of the wing shape of a UAV.
更多查看译文
关键词
adaptive kernels,robot control,active policy search,trial-and-error methodology,reinforcement learning,global optimization,Gaussian process,optimal decision making,Monte Carlo method,black-box Bayesian optimization,UAV
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要