Towards a conceptual framework for assessing the effectiveness of digital game-based learning

In recent years, interest has grown in the systematic assessment of the effectiveness of digital game-based learning (DGBL). A conceptual framework describing what effectiveness means in the context of DGBL and which are its subcomponents has hitherto been lacking however. Hence, the goal of this paper is to propose a conceptualization and operationalization of effectiveness rooted in social-cognitive theory. In order to identify desired outcomes and be able to operationalize effectiveness, focus groups were organized with three stakeholder groups following a user requirements analysis methodology.Results indicate that three categories of desired outcomes can be distinguished: learning, motivational and efficiency outcomes. For the different outcomes, different subcomponents can be extracted which can be organized hierarchically. Learning outcomes that are seen as relevant to the effectiveness of DGBL are 1) increased interest in the subject matter, 2) improvement in objective performance (e.g., in a test), and 3) transfer, referring to the player's ability to apply acquired knowledge or skills to real-world situations. Relevant motivational outcomes concern 1) enjoyment, the extent to which playing the game evoked an enjoyable experience, and 2) increased motivation to learn using DGBL. Efficiency outcomes relevant to DGBL effectiveness, finally, are related to 1) time management and 2) cost-effectiveness. Overall, it can be stated that a DGBL intervention is effective when it achieves similar or higher scores compared to other instructional methods in relation to any of the above mentioned outcomes without significantly (in the common, not the statistical sense) diminishing any of the others. An operationalization of digital game-based learning effectiveness is provided.The operationalization is based on desired outcomes of digital game-based learning.A user requirements analysis among relevant stakeholders groups was conducted.Learning, motivational and efficiency outcomes can be distinguished.