An Online Evolving Method For a Safe and Fast Automated Vehicle Control System

An online evolving method, named evolving finite state machine (e-FSM), is proposed to develop an optimal Markov driving model. The model has the same properties as a standard Markov model, but its states and transition dynamics evolve without human supervision. In this article, we introduce: 1) the principles of the e-FSM’s novel capabilities: online state determination and online transition-dynamics identification for elaborating the Markov driving model and 2) an advanced online evolving framework (a-OEF) for supporting the reinforcement-learning-based controller’s decision making by using the evolved model. For the evaluation of the proposed methodology and framework, the ego vehicle is controlled by the double deep ${Q}$ -network (DDQN) controller with and without the a-OEF in the multilane driving scenario where various naturalistic traffic situations are simulated. Simulation results show that better control performance in terms of fast and safe driving is achieved via the DDQN with the a-OEF, which demonstrates that the Markov driving models evolved by the e-FSMs effectively support detecting and revising the controller’s incorrect decision making.