Uncovering the Causes of Emotions in Software Developer Communication Using Zero-shot LLMs
CoRR(2023)
摘要
Understanding and identifying the causes behind developers' emotions (e.g.,
Frustration caused by `delays in merging pull requests') can be crucial towards
finding solutions to problems and fostering collaboration in open-source
communities. Effectively identifying such information in the high volume of
communications across the different project channels, such as chats, emails,
and issue comments, requires automated recognition of emotions and their
causes. To enable this automation, large-scale software engineering-specific
datasets that can be used to train accurate machine learning models are
required. However, such datasets are expensive to create with the variety and
informal nature of software projects' communication channels.
In this paper, we explore zero-shot LLMs that are pre-trained on massive
datasets but without being fine-tuned specifically for the task of detecting
emotion causes in software engineering: ChatGPT, GPT-4, and flan-alpaca. Our
evaluation indicates that these recently available models can identify emotion
categories when given detailed emotions, although they perform worse than the
top-rated models. For emotion cause identification, our results indicate that
zero-shot LLMs are effective at recognizing the correct emotion cause with a
BLEU-2 score of 0.598. To highlight the potential use of these techniques, we
conduct a case study of the causes of Frustration in the last year of
development of a popular open-source project, revealing several interesting
insights.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要