Designing Computational Evaluations for Graph Layout Algorithms: the State of the Art

Computational evaluations and benchmarks are widely used for validating graph and network layout algorithms (a.k.a. graph drawing). These evaluations can provide the reader with substantial insight into the performance and capabilities of the algorithm. Moreover, computational evaluations can help the reader determine whether the algorithm would fit a specific purpose, for example, graphs with many nodes or dense graphs. There is no standard approach for evaluating layout algorithms. Prior work holds a ``Wild West'' of diverse benchmark datasets and data characteristics, as well as varied evaluation metrics and ways to report results. It is often difficult to compare layout algorithms without first implementing them and then running your own evaluation. In this systematic review of past research, we examine the various methods used to run computational evaluations, the techniques used, the results reported, and the advantages and disadvantages of using one methodology over another. The proposed systematic review---and accompanying website---will guide readers through evaluation types, the types of results reported, and the available benchmark datasets and their data characteristics.