Multimodal prediction of head nods in dyadic conversations

Abstract

Non-verbal expressions in human interactions carry important messages. These messages, which constitute a significant part of the information to be transferred, are not used effectively by machines in human-robot/agent interaction. In this study, the purpose is to predict the potential head nod moments for robot/agent and therefore to develop more human-like interfaces. To achieve this, acoustic feature extraction and social signal annotations are carried out on human-human dyadic conversations. A certain history window for each head nod instances are fed to binary classification. Consequently, upon the classification by Support Vector Machines, 'potential head nod' or 'no head nod' outputs are obtained. More than half of the head nods are succesfully predicted as 'potential head nod', which leads promising results for human-like robot/agents.