Emotion driven Crisis Response: A benchmark Setup for Multi-lingual Emotion Analysis in Disaster Situations

Emotion analysis from texts has emerged as an important area of research in the field of Natural Language Processing (NLP) in the past few years. Several benchmark datasets have been released for this task, however most of the datasets are open domain in nature and for resource-rich language like English. These datasets may not always be sufficient for capturing domain specific emotions, and may tend to skew towards an emotion based on the domain. In this paper, we provide a framework for multilingual emotion detection to deal with the crisis situations. We collect and annotate disaster domain social media tweets and news data in two languages, namely English and Hindi. We derive 6 emotions from Plutchik's wheel of emotions suitable for disaster domain, and annotate the data using these disaster specific emotions. In total we create four emotionally enriched datasets i.e. 2 tweets datasets (English and Hindi) and 2 news dataset (English and Hindi). We also establish strong baselines on the dataset using two popular deep learning algorithms, stacked Bi-LSTM+CNN and BERT. Evaluation shows that the best model achieves the averaged accuracy of 68% across the four different datasets.