Realtime engagement measurement in human-computer interaction

Abstract

Social robots are expected to understand their interlocutors and behave accordingly like humans do. Endowing robots with the capability of monitoring user engagement during their interactions with humans is one of the crucial steps towards achieving this goal. In this work, an interactive game is designed and implemented, which is played with a robot. During the interaction, the user engagement is monitored in realtime via detection of user gaze, turn-taking, laughters/smiles and head nods from audio-visual data. In the experiments conducted, the real-time monitored engagement is found to be consistent with the human-annotated engagement levels.