Empirically Detecting False Test Alarms Using Association Rules
ICSE), 2015 IEEE/ACM 37th IEEE International Conference(2015)
摘要
Applying code changes to software systems and testing these code changes can be a complex task that involves many different types of software testing strategies, e.g. system and integration tests. However, not all test failures reported during code integration are hinting towards code defects. Testing large systems such as the Microsoft Windows operating system requires complex test infrastructures, which may lead to test failures caused by faulty tests and test infrastructure issues. Such false test alarms are particular annoying as they raise engineer attention and require manual inspection without providing any benefit. The goal of this work is to use empirical data to minimize the number of false test alarms reported during system and integration testing. To achieve this goal, we use association rule learning to identify patterns among failing test steps that are typically for false test alarms and can be used to automatically classify them. A successful classification of false test alarms is particularly valuable for product teams as manual test failure inspection is an expensive and time-consuming process that not only costs engineering time and money but also slows down product development. We evaluating our approach on system and integration tests executed during Windows 8.1 and Microsoft Dynamics AX development. Performing more than 10,000 classifications for each product, our model shows a mean precision between 0.85 and 0.90 predicting between 34% and 48% of all false test alarms.
更多查看译文
关键词
data mining,learning (artificial intelligence),operating systems (computers),program testing,software reliability,Microsoft Dynamics AX development,Microsoft Windows operating system,Windows 8.1,association rule learning,code defects,code integration,complex test infrastructures,false test alarm detection,integration tests,manual test failure inspection,software systems,software testing strategy,test failures
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络