Analysis of the Gaze Behavior of Deaf and Hard-of-Hearing Students During a Captioned Lecture.

Captions are key tools for allowing deaf and hard-of-hearing audiences to obtain aural (audio) information. In a classroom setting, people with normal hearing can listen to the lecture while referencing the white/blackboard and documents to effectively combine the information, whereas those with hearing impairments cannot. During lectures, people with hearing impairments need to move their eyes (gaze direction) more than those with normal hearing. The purpose of this study is to investigate the tendency and characteristics of gaze behavior of deaf and hard-of-hearing persons during captioned lectures. In the experiment, we analyzed the gaze behavior, in terms of the gaze count and duration, in relation to the following gaze targets (points of gaze): a lecturer with slides ( L ), handouts ( H ), and captions ( C ). Furthermore, we created hybrid captions, wherein parts of the slide were cut out and inserted between chunks of text to reduce eye movement (EM). The study compared the gaze behavior of subjects viewing the hybrid captions to that of subjects viewing normal text-only captions. The gaze behavior for both captions was essentially similar. The analysis of the findings showed that the gaze behavior was not cyclic; instead, it was roundtrip-type behavior centered around L , the source of the lecture content. Additionally, the study suggested that hybrid captions were seamlessly accepted and used in a similar manner as normal captions, even though they did not reduce fatigue caused by EMs.