Unsupervised Scalable Multimodal Driving Anomaly Detection

Driving anomaly detection aims to identify objects, events or actions that can increase the risk of accidents, reducing road safety. While supervised approaches can effectively identify aspects related to driving anomalies, it is unfeasible to tabulate and address all potential driving anomalies. Instead, it is appealing to design unsupervised approaches that can automatically identify unexpected driving scenarios. This study formulates the detection of driving anomalies as a binary-discrimination task between expected and unexpected driving behaviors. We propose an unsupervised contrastive method using conditional generative adversarial networks (GANs) implemented with the attention model and the triplet loss function. A feature of our framework is its scalability, where it is easy to add new modalities. We consider five different modalities: the vehicle's CAN-Bus signals, driver's physiological signals, distance to nearby pedestrians, distance to nearby vehicles and distance to nearby bicycles. Our approach trains a conditional GAN to extract latent features from each of the five modalities. An attention model combines the latent representations from the modalities. The entire framework is trained with the triplet loss function to generate effective representations to discriminate normal and abnormal driving segments. We conduct experimental evaluations on the driving anomaly dataset (DAD), achieving improved performance over alternative approaches.