Explanation videos available online can improve learning, sometimes

Students often search for answers online, and gravitate to short explanation videos, which are abundant online. While they seem useful and can be quite engaging, it is unclear if these videos benefit classroom learning. We hypothesized that when carefully matched to the class instruction, video can be helpful, but that benefits will depend on the ability of students to pay attention. To test this we recruited students from first- and second-year college physics classes (n=139) and showed them videos matching their course material. Quizzes before or after the videos measured the benefit of the video. We also measured eye movement to gauge attention and analyzed results as a function of grade point average (GPA). We saw a benefit of video on quiz performance for the average student (Cohen’s d=0.31 across quizzes, 0.45 best case), and larger effects for students with higher GPA. GPA did not predict attention, nor did attention consistently affect performance. In an online replication with participants of a broader STEM background (n=215) the benefit was apparent for only one of the four videos tested (Cohen’s d’=1.0). We conclude that some short explanation videos available online can be an effective support to classroom instruction. However, they benefit mostly students that are strong academically, despite similar levels of overt attention across all students.