How Many Abilities Can We Measure in Computational Thinking?: A Study on Bebras Challenge

While several approaches have been proposed to assess computational thinking (CT) abilities, it is still unclear how many and which these abilities are. Despite the undisputed importance of assessment, the fact is that there is not enough evidence on which abilities are merely theoretical and which can be empirically observed in the context of CT. This paper is part of a larger investigation in which we try to answer a simple question: can CT abilities be quantitatively defined and measured? In this particular study, we try to answer a simple question: How many CT abilities can be empirically observed using factor analysis? We approach this question, using a dataset containing answers of 1564 students from Lithuania in the Bebras Challenge from 2015. Firstly, we used confirmatory factor analysis to verify a theory that claims that five CT abilities are assessed by the contest. Our analysis shows the theory is not statistically supported. Secondly, we used principal component analysis as an exploratory analysis to try to derive an appropriate number of factors from the data. Surprisingly, the analysis suggests there are only two main recognizable factors. Finally, we briefly discuss these factors and hypothesize that the first is called evaluation ability, which would include abstraction, generalization, and decomposition, while factor 2 is algorithmic thinking and logical reasoning.