Impact of visual speech on gaze following in monolingual and bilingual adults

Altered sensory and language experience can shape visual attention to faces. Hearing bilingual infants show increased visual attention to the mouth compared to monolingual infants. It is unclear how this altered visual attention to the face influences the processing of other crucial social cues, such as eye gaze. The goal of this online study was to determine how visual speech and participant’s prior language experience influences gaze following. Monolingual (n=20) and bilingual (n=22) hearing native English speakers (21.4 ± 2.7 yrs) were tested. The second language of bilingual participants encompassed a range of other languages. Participants performed a peripheral target discrimination task, preceded by cues to the target location that had equal probability of being valid or invalid. Cues were either arrow cues (a non-social control) or eye-gaze cues in a concurrently viewed static face. For these social gaze conditions, gaze shifts were accompanied by either: i) closed neutral mouth, ii) speech cues (linguistic mouth movements), or iii) non-speech cues (non-linguistic mouth movements). The magnitude of the gaze cueing effect was quantified as the difference between response times (RTs) on correctly performed valid and invalid trials. As a potential correlate, the lipreading ability of the subjects was assessed separately using an open-response, consonant-vowel-consonant word test. Across all conditions, bilinguals demonstrated a smaller gaze-cueing effect than monolinguals. In bilingual participants only, lipreading proficiency positively correlated with the gaze cueing effect when visual speech cues were also present, but not in non-speech conditions. Our results show that language experience can impact attentional orienting to facial cues. The effect in bilingual subjects may reflect enhanced division and shifting of attention between the eyes and the mouth when linguistic information is present in those participants who are more fluent lipreaders.