On the Safety of Automotive Systems Incorporating Machine Learning Based Components: A Position Paper

Machine learning (ML) components are increasingly adopted in many automated systems. Their ability to learn and work with novel input/incomplete knowledge and their generalization capabilities make them highly desirable solutions for complex problems. This has motivated the inclusion of ML techniques/components in products for many industrial domains including automotive systems. Such systems are safety-critical systems since their failure may cause death or injury to humans. Therefore, their safety must be ensured before they are used in their operational environment. However, existing safety standards and Verification and Validation (V&V) techniques do not properly address the special characteristics of ML-based components such as non-determinism, non-transparency, instability. This position paper presents the authors' view on the safety of automotive systems incorporating ML-based components, and it is intended to motivate and sketch a research agenda for extending a safety standard, namely ISO 26262, to address challenges posed by incorporating ML-based components in automotive systems.