Assessing others' knowledge through their speech disfluencies and gestures

Abstract

As part of the multimodal language system, gestures play a vital role for listeners, by capturing attention and providing information. Similarly, disfluencies serve as a cue for the listeners about one's knowledge on a topic. In two studies, the first study with natural and the second study with controlled stimuli, we asked whether the combination of gestures and speech disfluencies would affect how listeners made feeling-of-another's-knowing (FOAK) judgments regarding speakers' knowledge states. In Study 1, we showed participants videos of speakers providing navigational instruction. We manipulated the speakers' use of gestures and speech disfluencies, whereas facial expressions, words, and additional visual cues (e.g., background, clothes, objects) naturally occurred. We found that fluent speech elicited higher FOAK ratings than disfluent speech, but no significant effect was found for gestures. In the follow-up Study 2, we examined the same disfluency-gesture interaction in a more controlled setting using video stimuli with an actress controlling for background, intonation, and word choice, as well as iconic and beat gesture types as gesture subcategories. Participants also filled out the Gesture Awareness Scale. Results were identical with the first study, in which only the disfluent speech received significantly lower FOAK ratings, revealing no effects of gesture use or type. These findings suggest that individuals may use certain communicative cues more than others, particularly in the context of assessing others' knowledge.