A Real-time Fatigue Driving Detection Method Based on Facial Multi-Feature Fusion

Fatigue driving refers to the imbalance of physiological and psychological functions after driving continuously for a long time, which leads to the objective decline of driving skills. Fatigue driving is one of the main factors leading to traffic accidents, and most accidents related to fatigue driving can be avoided. Effective real-time detection of driver fatigue state and timely early warning prompt can greatly reduce and prevent this kind of traffic accidents and reduce casualties and property losses. In this paper, we present a real-time fatigue detection method based on the driver’s facial features, taking the state of the eye and mouth as the main fatigue judgment basis. The algorithm consists of three parts: Facial Feature Location, State Recognition, and Fatigue State Estimation. Firstly, the eye and mouth regions are extracted by combining the improved MTCNN and PFLD. Then, a general two-classification network was designed to recognize the state of eyes and mouth. Finally, a sliding window model is designed to record the driver’s state information in each frame of image, and to judge the driver’s fatigue state by combining PERCLOS, blinking frequency and yawning frequency. We tested the performance of the algorithm based on video data set and camera input. An accuracy of 95% was achieved on the entire data set, and the average detection time is 38.72 ms. The experimental results show that the proposed method has good real-time performance and high accuracy rate.