Uncertainty-Aware Model-Based Offline Reinforcement Learning for Automated Driving

Offline reinforcement learning (RL) provides a framework for learning decision-making from offline data and therefore constitutes a promising approach for real-world applications such as automated driving (AD). Especially in safety-critical applications, interpretability and transferability are crucial to success. That motivates model-based offline RL approaches, which leverage planning. However, current state-of-the-art (SOTA) methods often neglect the influence of aleatoric uncertainty arising from the stochastic behavior of multi-agent systems. Further, while many algorithms state that they are suitable for AD, there is still a lack of evaluation in challenging scenarios. This work proposes a novel approach for Uncertainty-aware Model-Based Offline REinforcement Learning Leveraging pl A nning (UMBRELLA), which jointly solves the prediction, planning, and control problem of the self-driving vehicle (SDV) in an interpretable learning-based fashion. A trained action-conditioned stochastic dynamics model captures distinctively different future evolutions of the traffic scene. The analysis provides empirical evidence for the effectiveness of our approach and SOTA performance in challenging AD simulations and using a real-world public dataset.