Safe Reinforcement Learning Benchmark Environments for Aerospace Control Systems
2022 IEEE Aerospace Conference (AERO)(2022)
摘要
Recent advancements in reinforcement learning techniques demonstrate an ability to make decisions in high dimen-sional state spaces and complex real-time strategy games. In contrast to supervised learning which features large data sets, there are relatively few existing environments for training rein-forcement learning agents. In addition, small differences in re-wards or action spaces can drastically change the difficulty and results of the training environments. Benchmarks seek to tackle both of these challenges by creating common environments, in the form of “Gyms” to train and compare reinforcement learning techniques, approaches, and algorithms. Many gyms, such as the classical control and Atari games environments, have become standard in new research on reinforcement learning. Researchers can easily compare and benchmark competing so-lutions across publications on these universal baselines enabling rapid innovation and collaboration. However, there are currently no standard set of environments for aerospace problems, and many of the gyms in the literature do not include safety con-straints or run time assurance systems that intervene when the reinforcement learning agent violates safety constraints. This manuscript describes the development of the Aerospace SafeRL Framework and accompanying Aerospace SafeRL Benchmarks that include interactive environments, safety constraints, soft-ware interfaces for run time assurance safety monitors with base implementations, and an initial set of baseline solutions. This initial set of scenarios introduces simple RL environments that expose the kinds of motion patterns, dynamics, and safety constraints encountered in air and space problems in 2D and 3D. This manuscript also describes standardized evaluation metrics for these environments to provide a consistent performance measurement with aerospace relevance. These benchmarks pro-vide a structured foundation for future reinforcement learning algorithms, run time assurance designs, and neural network verification techniques for the aerospace domain.
更多查看译文
关键词
neural network verification techniques,consistent performance measurement,standardized evaluation metrics,motion patterns,software interfaces,complex real-time strategy games,decision making,aerospace domain,run time assurance designs,reinforcement learning algorithms,space problems,simple RL environments,run time assurance safety monitors,interactive environments,Aerospace SafeRL Framework,safety constraints,reinforcement learning agent,run time assurance systems,aerospace problems,standard set,Atari games environments,classical control,gyms,training environments,reinforcement learning agent training,supervised learning,high dimensional state spaces,reinforcement learning techniques,Aerospace control systems,safe reinforcement learning benchmark environments
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要