Analyzing Mobile Application Usage: Generating Log Files From Mobile Screen Recordings

Logging mobile application usage on smartphones is limited to rather general system events unless one has access to the operating system's or applications' source code. In this paper, we present a method for analyzing mobile application usage in detail by generating log files based on mobile screen output. We are combining long-term log file analysis and short-term screen recording analysis by utilizing existing computer vision and machine learning methods. To validate the log results of our approach and implementation we collect 118 sample screen recordings of phone usage sessions and evaluate the resulting log file manually. Besides that, we explore the performance of our approach with different video quality parameters: frame rate and bit rate. We show that our method provides detailed data about application use and can work with low-quality video under certain circumstances.