Does modality matter? A meta-analysis of the effect of video input in L2 listening assessment

Over the past two decades, there has been a growing research interest in examining the effects of video-based second language (L2) listening tests. However, these studies have shown inconsistency in their findings, prompting the need for a comprehensive analysis. This study aims to address this issue by conducting a meta-analysis to synthesize the quantitative research in this field. Our primary objective is to determine the pooled effect of video-based L2 listening assessment on test-takers' listening comprehension and identify moderators that could potentially influence this effect. These potential moderators encompassed participants' characteristics, research method, video input features, and outcome measures. Through an extensive search process, we identified a total of 28 primary studies (years 1984-2022) that contributed data from 43 independent samples. We found that video input had a small overall positive effect on test -takers' performance (g = 0.297). Additionally, we observed patterns in the effect of video input across different levels of moderators such as test-takers' language proficiency, research design, reporting of reliability, speaker presentation, video length, question accessibility, note-taking availability, and item format. We discuss the implications of these findings and conclude with the limitations and several perspectives for future research.