BenchBuild : A Large-Scale Empirical-Research Toolkit

The efficiency of software is commonly evaluated with one or more suites of experiments at compile time or run time. The manual preparation of such experiments is tedious and error-prone, involving tasks such as intercepting the compiler at hand or the resulting binaries with custom measurements. This imposes a practical limit on the number of case studies. We present BenchBuild, a large-scale empirical-research toolkit that supports 18978 projects for compile-time and 188 projects for run-time testing. BenchBuild automates most of the tasks involved and provides tools that reduce the amount of effort required to increase the test coverage. https://youtu.be/VtXdveMT1Rk CCS Concepts •Software and its engineering → Software usability; Software testing and debugging; Extra-functional properties; Preprocessors;