Too Simple? Notions of Task Complexity used in Maintenance-based Studies of Programming Tools.

Researchers conducting studies on programming tools often make use of maintenance tasks. The complexity of these tasks can significantly influence how participants behave. At the same time, the complexity of tasks is difficult to pinpoint due to the many sources of complexity for maintenance tasks. As a result, researchers may struggle to deliberately decide in which regard their tasks should be complex and in which regard they should be simple. To help researchers deliberately influence task complexity, we discuss different factors of task complexity. We draw these factors from 23 selected and 39 surveyed studies on programming tools. We arrange the factors according to a task complexity model from ergonomics research that we adapt for maintenance tasks. We illustrate the application of the factors through an example critique of a task design. In the end, task complexity might always be too complex to be fully controlled. Nevertheless, we hope that our discussion helps other researchers to decide in which dimensions their tasks are complex and in which dimensions they want to keep them simple.