Sentiment Pen: Recognizing Emotional Context Based on Handwriting Features

In this paper, we discuss the assessment of the emotional state of the user from digitized handwriting for implicit human-computer interaction. The proposed concept exemplifies how a digital system could recognize the emotional context of the interaction. We discuss our approach to emotion recognition and the underlying neurophysiological mechanisms. To verify the viability of our approach, we have conducted a series of tests where participants were asked to perform simple writing tasks after being exposed to a series of emotionally-stimulating video clips from EMDB[6], one set of four clips per each quadrant on the circumplex model of emotion[28]. The user-independent Support Vector Classifier (SVC) built using the recorded data shows up to 66% accuracy for certain types of writing tasks for 1 in 4 classification (1. High Valence, High Arousal; 2. High Valence, Low Arousal; 3. Low Valence, High Arousal; 4. Low Valence, Low Arousal). In the same conditions, a user-dependent classifier reaches an average of 70% accuracy across all 12 study participants. While future work is required to improve the classification rate, this work should be seen as proof-of-concept for emotion assessment of users while handwriting aiming to motivate research on implicit interaction while writing to enable emotion-sensitivity in mobile and ubiquitous computing.