Performance evaluation of complex multi-thread applications through execution path analysis

The sporadic latency problems of multi-threaded applications remain a major concern for developers. A performance issue may be caused by internal factors such as an error in the code, inadequate database design or external factors such as thread imbalances, resource contention or system overload. In complex applications such as the Chromium browser, which is our focus in this paper, being able to collect precise unified information from several layers of the application or system is the first challenge. There exists a built-in tracer for Chromium, however, it provides a limited view of the application execution. The correlated analysis of the data collected from the multiple execution levels of the application is another challenge. In this paper, after explaining our tracing method for collecting data from the application and the operating system kernel simultaneously, we translate the multi-level tracing data into a unified model. We then use this model to perform kernel-level control flow analysis for user-level browser actions, to detect latency issues and identify their main causes. We show the effectiveness of the proposed method by detecting three Chromium performance bugs and finding their root-causes. Although our use-cases are based on the Chromium browser, the proposed method and lessons learned could be adapted for analysis of other complex multi-threaded applications.