Teeth-Brushing Recognition Based On Deep Learning

In this paper, we propose a multi-stream deep learning framework to tackle the activity recognition problem of sixteen kinds of Bass brushing methods by brushing photos and sensor data. This is a challenge task because the model needs to infer the relevance between images and sensor data. In order to solve this problem, CNN model is exploited to learn the spatial features from images and the LSTM model is used to learn the temporal features from sensor data. Then, a fusion scheme is proposed for prediction. Experimental results show that our model achieves high accuracy by using both images and sensor data under the constraint that the dataset is still quite limited.