Visual-spatial and verbal abilities differentially affect processing of gestural vs. spoken expressions

Abstract

Listeners are sensitive to speakers' co-speech iconic gestures. Concurrent visual and verbal information compete for attentional resources during multimodal comprehension. The current study examined the role of individual differences in visual-spatial vs. verbal abilities on individuals' differential sensitivity to gestural vs. spoken expressions. Turkish-speaking adults (N = 83) were tested on their sensitivity to concurrent gesture vs. speech in an online task (Kelly, S. D., ozyurek, A., & Maris, E. (2010a). Two sides of the same coin. Psychological Science, 21(2), 260-267) and were administered spatial and verbal working memory measures. Participants were slower and less accurate when gesture and speech were incongruent to one another compared to the baseline condition, in which they expressed congruent information. People with higher spatial working memory capacity were more efficient in processing gestures whereas people with higher verbal working memory capacity were more sensitive to spoken expressions. These suggest that not all people are equally sensitive to co-speech gestures and some people may benefit more from gestures during comprehension.