On the Pursuit of Developer Happiness: Webcam-Based Eye Tracking and Affect Recognition in the IDE

Recent research highlights the viability of webcam-based eye tracking as a low-cost alternative to dedicated remote eye trackers. Simultaneously, research shows the importance of understanding emotions of software developers, where it was found that emotions have significant effects on productivity, code quality, and team dynamics. In this paper, we present our work towards an integrated eye-tracking and affect recognition tool for use during software development. This combined approach could enhance our understanding of software development by combining information about the code developers are looking at, along with the emotions they experience. The presented tool utilizes an unmodified webcam to capture video of software developers while interacting with code. The tool passes each frame to two modules, an eye tracking module that estimates where the developer is looking on the screen, and an affect recognition module that infers developer emotion from their facial expressions. The proposed work has implications to researchers, educators, and practitioners, and we discuss some potential use cases in this paper.