Retellings of emotional videos: the dynamics of gesture characteristics in diverse emotion categories

Abstract

We express and ground our emotions through multiple channels. Across these channels, gestures play a critical role in emotion communication, yet how characteristics of gestures change across different emotion categories is still not well understood. We investigated the gesture characteristics (i.e. frequency, type, hand use, location, and direction) when individuals talk about emotional content. In a within-subject design, speakers (n = 36) described video clips representing happiness, anger, sadness, and neutral emotions. Our results showed that more representational gestures were produced while retelling videos for happiness and neutral than anger and sadness. Participants used both hands more frequently than their left hands for the retelling of happiness and used both hands more frequently than the left or right hands for the retelling of anger. Furthermore, gestures were predominantly performed at the centre of the body across happiness, anger, and neutral categories, with a notable preference for the vertical axis in the happiness category. Overall, these results suggest that specific gesture characteristics can be reflected in different emotion categories.