Common and differential acoustic representation of interpersonal and tactile iconic perception of Mandarin vowels

Many have found that sound sequences can convey various domains of meaning, yet the mechanisms underlying such apparent associations between particular sound sequences and meanings in speech are underexplored. The study aimed to examine the common and different acoustic cues crucial for tactile and social sound symbolism based on the vowels in Mandarin. With the full spectrum of vowels tested, the study provided rich sound variations to demonstrate the role of multiple acoustic cues in encoding different meanings at the same time. Forty native Chinese listened to vowel sequences and rated on the interpersonal (e.g., encouraging-authoritative) and tactile (e.g., light-heavy) attributes based on these sounds. Acoustic analysis was performed on the vowel sequences and XGBoost algorithm was further applied to simulate tactile/social perception. The results showed that duration and pitch are two features crucial for representing both tactile and social sound symbolism. The second formant and nasality of vowels are specifically important for tactile representation while the third formant is fundamental for social representation. Machine learning models can achieve an above-chance performance using limited yet important acoustic features of vowels. Findings of this study have implications for multi-modal speech perception and iconic sound-meaning mappings in Mandarin.